Compare commits

..
Author SHA1 Message Date
Zhang Yuxuan f3e6825d0c Merge branch 'master' into fix-dispatch-error 2025-03-15 21:41:25 +08:00
tomsun28 e7e38f3aa3 [bugfix] fix collect dispatch error
Signed-off-by: tomsun28 <tomsun28@outlook.com>
2025-03-15 18:54:06 +08:00
615 changed files with 5954 additions and 18456 deletions
-36
View File
@@ -2325,42 +2325,6 @@
"contributions": [
"code"
]
},
{
"login": "JuJinPark",
"name": "jujin",
"avatar_url": "https://avatars.githubusercontent.com/u/44892459?v=4",
"profile": "https://github.com/JuJinPark",
"contributions": [
"code"
]
},
{
"login": "LL-LIN",
"name": "LL-LIN",
"avatar_url": "https://avatars.githubusercontent.com/u/43002118?v=4",
"profile": "https://github.com/LL-LIN",
"contributions": [
"code"
]
},
{
"login": "bigcyy",
"name": "Yang Chen",
"avatar_url": "https://avatars.githubusercontent.com/u/73413979?v=4",
"profile": "https://bigcyy.github.io/",
"contributions": [
"code"
]
},
{
"login": "sarthakeash",
"name": "Sarthak Arora",
"avatar_url": "https://avatars.githubusercontent.com/u/74091160?v=4",
"profile": "https://github.com/sarthakeash",
"contributions": [
"code"
]
}
],
"contributorsPerLine": 7,
-2
View File
@@ -17,9 +17,7 @@
/home export-ignore
/hip export-ignore
/template-marketplace export-ignore
/.github export-ignore
/.idea export-ignore
/.devcontainer export-ignore
.gitattributes export-ignore
.gitignore export-ignore
-2
View File
@@ -31,8 +31,6 @@ name: "CodeQL"
on:
schedule:
- cron: '21 13 * * 4'
push:
branches: [ action* ]
jobs:
analyze:
+1 -1
View File
@@ -19,7 +19,7 @@ name: DOC CI
on:
push:
branches: [ master, dev, action* ]
branches: [ master, dev ]
paths:
- 'home/**'
pull_request:
+1 -2
View File
@@ -17,8 +17,7 @@
name: 'HertzBeat Doc Pdf Builder'
on:
push:
branches: [ page_build* ]
page_build
jobs:
build:
+1 -1
View File
@@ -22,7 +22,7 @@ name: Frontend CI
on:
push:
branches: [ master, dev, action* ]
branches: [ master, dev ]
paths:
- 'web-app/**'
pull_request:
+1 -1
View File
@@ -22,7 +22,7 @@ name: License Checker
on:
push:
branches: [ master, dev, action* ]
branches: [ master, dev ]
pull_request:
branches: [ master, dev ]
+1 -1
View File
@@ -42,7 +42,7 @@ jobs:
run: |
mvn clean install
mvn clean package -Prelease -DskipTests
cd hertzbeat-collector
cd collector
mvn clean package -Pcluster -DskipTests
- uses: docker/setup-qemu-action@v3
-3
View File
@@ -13,7 +13,6 @@ mvnw.bat
package-lock.json
*.zip
jdk/
data/
### STS ###
.apt_generated
@@ -55,5 +54,3 @@ node_modules
application-dev.yml
application-mysql.yml
application-pg.yml
/package.json
/yarn.lock
+1 -1
View File
@@ -1,5 +1,5 @@
Apache HertzBeat (incubating)
Copyright 2024-2025 The Apache Software Foundation
Copyright 2024 The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).
-6
View File
@@ -510,12 +510,6 @@ Thanks to these wonderful people, welcome to join us:
<td align="center" valign="top" width="14.28%"><a href="https://github.com/yasminvo"><img src="https://avatars.githubusercontent.com/u/107528848?v=4?s=100" width="100px;" alt="yasminvo"/><br /><sub><b>yasminvo</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=yasminvo" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/notbugggg"><img src="https://avatars.githubusercontent.com/u/147966331?v=4?s=100" width="100px;" alt="不关银渐层的事哦"/><br /><sub><b>不关银渐层的事哦</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=notbugggg" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/yyahang"><img src="https://avatars.githubusercontent.com/u/90464876?v=4?s=100" width="100px;" alt="yyahang"/><br /><sub><b>yyahang</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=yyahang" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/JuJinPark"><img src="https://avatars.githubusercontent.com/u/44892459?v=4?s=100" width="100px;" alt="jujin"/><br /><sub><b>jujin</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=JuJinPark" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/LL-LIN"><img src="https://avatars.githubusercontent.com/u/43002118?v=4?s=100" width="100px;" alt="LL-LIN"/><br /><sub><b>LL-LIN</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=LL-LIN" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://bigcyy.github.io/"><img src="https://avatars.githubusercontent.com/u/73413979?v=4?s=100" width="100px;" alt="Yang Chen"/><br /><sub><b>Yang Chen</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=bigcyy" title="Code">💻</a></td>
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/sarthakeash"><img src="https://avatars.githubusercontent.com/u/74091160?v=4?s=100" width="100px;" alt="Sarthak Arora"/><br /><sub><b>Sarthak Arora</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=sarthakeash" title="Code">💻</a></td>
</tr>
</tbody>
</table>
-6
View File
@@ -509,12 +509,6 @@ Thanks these wonderful people, welcome to join us:
<td align="center" valign="top" width="14.28%"><a href="https://github.com/yasminvo"><img src="https://avatars.githubusercontent.com/u/107528848?v=4?s=100" width="100px;" alt="yasminvo"/><br /><sub><b>yasminvo</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=yasminvo" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/notbugggg"><img src="https://avatars.githubusercontent.com/u/147966331?v=4?s=100" width="100px;" alt="不关银渐层的事哦"/><br /><sub><b>不关银渐层的事哦</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=notbugggg" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/yyahang"><img src="https://avatars.githubusercontent.com/u/90464876?v=4?s=100" width="100px;" alt="yyahang"/><br /><sub><b>yyahang</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=yyahang" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/JuJinPark"><img src="https://avatars.githubusercontent.com/u/44892459?v=4?s=100" width="100px;" alt="jujin"/><br /><sub><b>jujin</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=JuJinPark" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/LL-LIN"><img src="https://avatars.githubusercontent.com/u/43002118?v=4?s=100" width="100px;" alt="LL-LIN"/><br /><sub><b>LL-LIN</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=LL-LIN" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://bigcyy.github.io/"><img src="https://avatars.githubusercontent.com/u/73413979?v=4?s=100" width="100px;" alt="Yang Chen"/><br /><sub><b>Yang Chen</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=bigcyy" title="Code">💻</a></td>
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/sarthakeash"><img src="https://avatars.githubusercontent.com/u/74091160?v=4?s=100" width="100px;" alt="Sarthak Arora"/><br /><sub><b>Sarthak Arora</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=sarthakeash" title="Code">💻</a></td>
</tr>
</tbody>
</table>
-5
View File
@@ -73,11 +73,6 @@
"field": "successCode",
"type": 4,
"paramValue": "200, 201"
},
{
"field": "enableUrlEncoding",
"type": 1,
"paramValue": true
}
]
}
@@ -17,39 +17,30 @@
package org.apache.hertzbeat.alert.config;
import jakarta.validation.constraints.NotBlank;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* Alibaba Cloud SMS configuration properties
*/
@Data
@NoArgsConstructor
@AllArgsConstructor
public class AlibabaSmsProperties {
/**
* Alibaba Cloud access key id
*/
@NotBlank(message = "AccessKeyId cannot be empty")
private String accessKeyId;
/**
* Alibaba Cloud access key secret
*/
@NotBlank(message = "AccessKeySecret cannot be empty")
private String accessKeySecret;
/**
* SMS signature
*/
@NotBlank(message = "SignName cannot be empty")
private String signName;
/**
* SMS template Code
*/
@NotBlank(message = "TemplateCode cannot be null")
private String templateCode;
}
@@ -17,7 +17,6 @@
package org.apache.hertzbeat.alert.config;
import jakarta.validation.constraints.NotBlank;
import lombok.Data;
/**
@@ -28,19 +27,16 @@ public class AwsSmsProperties {
/**
* AWS Cloud access key id
*/
@NotBlank(message = "AccessKeyId cannot be empty")
private String accessKeyId;
/**
* AWS Cloud access key secret
*/
@NotBlank(message = "AccessKeySecret cannot be empty")
private String accessKeySecret;
/**
* AWS Cloud region
*/
@NotBlank(message = "region cannot be empty")
private String region;
}
@@ -17,10 +17,7 @@
package org.apache.hertzbeat.alert.config;
import jakarta.validation.constraints.NotBlank;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
@@ -28,8 +25,6 @@ import org.springframework.stereotype.Component;
* SMS configuration
*/
@Data
@AllArgsConstructor
@NoArgsConstructor
@Component
@ConfigurationProperties(prefix = "alerter.sms")
public class SmsConfig {
@@ -42,7 +37,6 @@ public class SmsConfig {
/**
* sms service provider
*/
@NotBlank(message = "Type cannot be empty")
private String type;
/**
@@ -64,12 +58,6 @@ public class SmsConfig {
* Aws configuration
*/
private AwsSmsProperties aws;
/**
* Twilio SMS configuration
*/
private TwilioSmsProperties twilio;
/**
* Smslocal SMS configuration
*/
@@ -18,16 +18,12 @@
package org.apache.hertzbeat.alert.config;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* Smslocal SMS Properties
*/
@Data
@NoArgsConstructor
@AllArgsConstructor
public class SmslocalSmsProperties {
/**
* SmsLocal account api key
@@ -17,45 +17,35 @@
package org.apache.hertzbeat.alert.config;
import jakarta.validation.constraints.NotBlank;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* Tencent Cloud SMS properties
*/
@Data
@NoArgsConstructor
@AllArgsConstructor
public class TencentSmsProperties {
/**
* Tencent cloud account secret id
*/
@NotBlank(message = "SecretId cannot be empty")
private String secretId;
/**
* Tencent cloud account secret key
*/
@NotBlank(message = "SecretKey cannot be empty")
private String secretKey;
/**
* SMS app id
*/
@NotBlank(message = "AppId cannot be null")
private String appId;
/**
* SMS signature
*/
@NotBlank(message = "SignName cannot be empty")
private String signName;
/**
* SMS template ID
*/
@NotBlank(message = "templateId cannot be null")
private String templateId;
}
@@ -17,22 +17,16 @@
package org.apache.hertzbeat.alert.config;
import jakarta.validation.constraints.NotBlank;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* UniSMS properties
*/
@Data
@NoArgsConstructor
@AllArgsConstructor
public class UniSmsProperties {
/**
* UniSMS access key id
*/
@NotBlank(message = "accessKeyId cannot be empty")
private String accessKeyId;
/**
@@ -43,18 +37,15 @@ public class UniSmsProperties {
/**
* SMS signature
*/
@NotBlank(message = "signature cannot be null")
private String signature;
/**
* SMS template ID
*/
@NotBlank(message = "templateId cannot be null")
private String templateId;
/**
* Authentication mode: simple or hmac, default is simple
*/
@NotBlank(message = "authMode cannot be null")
private String authMode = "simple";
}
@@ -118,11 +118,11 @@ public class TencentCloudExternAlert {
/**
* Transaction alarm
*/
public static final String EVENT = "event";
private static final String EVENT = "event";
/**
* Indicator alarm
*/
public static final String METRIC = "metric";
private static final String METRIC = "metric";
}
@@ -48,7 +48,6 @@ final class TelegramBotAlertNotifyHandlerImpl extends AbstractAlertNotifyHandler
TelegramBotNotifyDTO notifyBody = TelegramBotNotifyDTO.builder()
.chatId(receiver.getTgUserId())
.text(renderContent(noticeTemplate, alert))
.messageThreadId(receiver.getTgMessageThreadId())
.disableWebPagePreview(true)
.build();
HttpHeaders headers = new HttpHeaders();
@@ -85,8 +84,6 @@ final class TelegramBotAlertNotifyHandlerImpl extends AbstractAlertNotifyHandler
private String text;
@JsonProperty("disable_web_page_preview")
private Boolean disableWebPagePreview;
@JsonProperty("message_thread_id")
private String messageThreadId;
}
@NoArgsConstructor
@@ -23,7 +23,6 @@ import org.apache.hertzbeat.alert.config.SmsConfig;
import org.apache.hertzbeat.alert.service.impl.SmsLocalSmsClientImpl;
import org.apache.hertzbeat.alert.service.impl.AwsSmsClientImpl;
import org.apache.hertzbeat.alert.service.impl.TencentSmsClientImpl;
import org.apache.hertzbeat.alert.service.impl.TwilioSmsClientImpl;
import org.apache.hertzbeat.alert.service.impl.UniSmsClientImpl;
import org.apache.hertzbeat.alert.service.impl.AlibabaSmsClientImpl;
import org.apache.hertzbeat.base.dao.GeneralConfigDao;
@@ -36,7 +35,6 @@ import org.springframework.stereotype.Component;
import static org.apache.hertzbeat.common.constants.SmsConstants.ALIBABA;
import static org.apache.hertzbeat.common.constants.SmsConstants.AWS;
import static org.apache.hertzbeat.common.constants.SmsConstants.TENCENT;
import static org.apache.hertzbeat.common.constants.SmsConstants.TWILIO;
import static org.apache.hertzbeat.common.constants.SmsConstants.UNISMS;
import static org.apache.hertzbeat.common.constants.SmsConstants.SMSLOCAL;
@@ -143,12 +141,9 @@ public class SmsClientFactory {
case AWS:
currentSmsClient = new AwsSmsClientImpl(smsConfig.getAws());
break;
case TWILIO:
currentSmsClient = new TwilioSmsClientImpl(smsConfig.getTwilio());
break;
default:
log.warn("[SmsClientFactory] Unsupported SMS provider type: {}", smsConfig.getType());
break;
}
}
}
}
@@ -18,22 +18,14 @@
package org.apache.hertzbeat.alert.service.impl;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.hertzbeat.alert.dto.TencentCloudExternAlert;
import org.apache.hertzbeat.alert.reduce.AlarmCommonReduce;
import org.apache.hertzbeat.alert.service.ExternAlertService;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.alerter.SingleAlert;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
/**
* Default external alarm service impl
*/
@@ -51,7 +43,8 @@ public class TencentExternAlertService implements ExternAlertService {
log.warn("parse extern alert content failed! content: {}", content);
return;
}
SingleAlert alert = new TencentCloudAlertConverter().convert(report);
// todo convert TenCloudAlertReport to SingleAlert
SingleAlert alert = new SingleAlert();
alarmCommonReduce.reduceAndSendAlarm(alert);
}
@@ -59,169 +52,4 @@ public class TencentExternAlertService implements ExternAlertService {
public String supportSource() {
return "tencent";
}
/**
* Converter: TencentCloud alert to SingleAlert
*/
public static class TencentCloudAlertConverter {
/**
* Metric alert content template
*/
private final String metricTemplate = "Tencent Cloud Metric Alert: {metricShowName} {calcType} {calcValue}{calcUnit}, Current: {currentValue}{unit}";
/**
* Event alert content template
*/
private final String eventTemplate = "Tencent Cloud Event Alert: {productShowName} - {eventShowName}";
/**
* Convert TencentCloud alert to SingleAlert
*/
public SingleAlert convert(TencentCloudExternAlert tencentAlert) {
// Building basic information
SingleAlert.SingleAlertBuilder builder = SingleAlert.builder()
.status(convertStatus(tencentAlert.getAlarmStatus()))
.startAt(parseTime(tencentAlert.getFirstOccurTime()))
.activeAt(parseTime(tencentAlert.getFirstOccurTime()))
.triggerTimes(1);
// If it is a recovery state, set the end time.
if ("0".equals(tencentAlert.getAlarmStatus())) {
builder.endAt(parseTime(tencentAlert.getRecoverTime()));
}
// Build labels
Map<String, String> labels = new HashMap<>();
buildLabels(labels, tencentAlert);
// Build annotations
Map<String, String> annotations = new HashMap<>();
buildAnnotations(annotations, tencentAlert);
// Build content
String content = generateContent(tencentAlert);
return builder
.labels(labels)
.annotations(annotations)
.content(content)
.build();
}
private void buildLabels(Map<String, String> labels, TencentCloudExternAlert alert) {
labels.put("__source__", "tencent_cloud");
labels.put("alert_type", alert.getAlarmType());
if (TencentCloudExternAlert.METRIC.equals(alert.getAlarmType())) {
labels.put("metric_name", alert.getAlarmPolicyInfo().getConditions().getMetricName());
labels.put("namespace", alert.getAlarmObjInfo().getNamespace());
} else {
labels.put("event_name", alert.getAlarmPolicyInfo().getConditions().getEventName());
labels.put("product_name", alert.getAlarmPolicyInfo().getConditions().getProductName());
}
}
private void buildAnnotations(Map<String, String> annotations, TencentCloudExternAlert alert) {
TencentCloudExternAlert.AlarmPolicyInfo alarmPolicyInfo = alert.getAlarmPolicyInfo();
TencentCloudExternAlert.AlarmObjInfo alarmObjInfo = alert.getAlarmObjInfo();
TencentCloudExternAlert.Dimensions dimensions = alert.getAlarmObjInfo().getDimensions();
putIfNotNull(annotations, "policy_id", alarmPolicyInfo.getPolicyId());
putIfNotNull(annotations, "policy_type", alarmPolicyInfo.getPolicyType());
putIfNotNull(annotations, "policy_name", alarmPolicyInfo.getPolicyName());
putIfNotNull(annotations, "policy_type_cname", alarmPolicyInfo.getPolicyTypeCname());
putIfNotNull(annotations, "namespace", alarmObjInfo.getNamespace());
putIfNotNull(annotations, "region", alarmObjInfo.getRegion());
putIfNotNull(annotations, "app_id", alarmObjInfo.getAppId());
putIfNotNull(annotations, "uin", alarmObjInfo.getUin());
putIfNotNull(annotations, "instance_id", dimensions.getUnInstanceId());
putIfNotNull(annotations, "obj_id", dimensions.getObjId());
}
/**
* Generate alert content with template
*/
private String generateContent(TencentCloudExternAlert alert) {
if (TencentCloudExternAlert.METRIC.equals(alert.getAlarmType())) {
return generateMetricContent(alert);
} else {
return generateEventContent(alert);
}
}
/**
* Generate metric alert content
*/
private String generateMetricContent(TencentCloudExternAlert alert) {
TencentCloudExternAlert.Conditions conditions = alert.getAlarmPolicyInfo().getConditions();
Map<String, String> params = new HashMap<>();
params.put("metricShowName", conditions.getMetricShowName());
if (StringUtils.isNotEmpty(conditions.getCalcType())
&& StringUtils.isNotEmpty(conditions.getCalcValue())) {
params.put("calcType", conditions.getCalcType());
params.put("calcValue", conditions.getCalcValue());
params.put("calcUnit", Objects.toString(conditions.getCalcUnit(), ""));
} else {
params.put("calcType", "");
params.put("calcValue", "N/A");
params.put("calcUnit", "");
}
if (StringUtils.isNotEmpty(conditions.getCurrentValue())) {
params.put("currentValue", conditions.getCurrentValue());
params.put("unit", Objects.toString(conditions.getUnit(), ""));
} else {
params.put("currentValue", "N/A");
params.put("unit", "");
}
return replacePlaceholders(metricTemplate, params);
}
/**
* Generate event alert content
*/
private String generateEventContent(TencentCloudExternAlert alert) {
TencentCloudExternAlert.Conditions conditions = alert.getAlarmPolicyInfo().getConditions();
Map<String, String> params = new HashMap<>();
params.put("productShowName", conditions.getProductShowName());
params.put("eventShowName", conditions.getEventShowName());
return replacePlaceholders(eventTemplate, params);
}
/**
* Replace placeholders in template with actual values
*/
private String replacePlaceholders(String template, Map<String, String> params) {
String result = template;
for (Map.Entry<String, String> entry : params.entrySet()) {
result = result.replace("{" + entry.getKey() + "}", Objects.toString(entry.getValue(), "NULL"));
}
return result;
}
private String convertStatus(String alarmStatus) {
return "1".equals(alarmStatus) ? CommonConstants.ALERT_STATUS_FIRING : CommonConstants.ALERT_STATUS_RESOLVED;
}
private Long parseTime(String timeStr) {
try {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
return sdf.parse(timeStr).getTime();
} catch (ParseException e) {
log.error("Failed to parse time: {}", timeStr);
throw new IllegalArgumentException("Failed to parse time: " + timeStr, e);
}
}
private void putIfNotNull(Map<String, String> map, String key, String value){
if (StringUtils.isNotEmpty(value)){
map.put(key, value);
}
}
}
}
@@ -17,7 +17,6 @@
package org.apache.hertzbeat.alert.service.impl;
import jakarta.xml.bind.DatatypeConverter;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.alert.config.TencentSmsProperties;
import org.apache.hertzbeat.alert.service.SmsClient;
@@ -35,6 +34,7 @@ import org.apache.http.util.EntityUtils;
import org.apache.hertzbeat.common.util.JsonUtil;
import com.fasterxml.jackson.databind.JsonNode;
import javax.xml.bind.DatatypeConverter;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.text.SimpleDateFormat;
@@ -1,171 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.alert.service.impl;
import com.fasterxml.jackson.databind.JsonNode;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.alert.config.TwilioSmsProperties;
import org.apache.hertzbeat.alert.service.SmsClient;
import org.apache.hertzbeat.common.entity.alerter.GroupAlert;
import org.apache.hertzbeat.common.entity.alerter.NoticeReceiver;
import org.apache.hertzbeat.common.entity.alerter.NoticeTemplate;
import org.apache.hertzbeat.common.support.exception.SendMessageException;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import java.net.URI;
import java.util.ArrayList;
import java.util.Base64;
import java.util.List;
import static org.apache.hertzbeat.common.constants.SmsConstants.TWILIO;
/**
* Twilio SMS Client Implementation<br>
* API doc: <a href=
* "https://www.twilio.com/docs/sms/api">https://www.twilio.com/docs/sms/api</a>
*/
@Slf4j
public class TwilioSmsClientImpl implements SmsClient {
private static final String API_URL_FORMAT = "https://api.twilio.com/2010-04-01/Accounts/%s/Messages.json";
private static final String MESSAGE_TEMPLATE = "Instance: %s, Priority: %s, Content: %s";
private final String accountSid;
private final String authToken;
private final String twilioPhoneNumber;
public TwilioSmsClientImpl(TwilioSmsProperties config) {
if (config != null) {
this.accountSid = config.getAccountSid();
this.authToken = config.getAuthToken();
this.twilioPhoneNumber = config.getTwilioPhoneNumber();
} else {
this.accountSid = "";
this.authToken = "";
this.twilioPhoneNumber = "";
}
}
@Override
public void sendMessage(NoticeReceiver receiver, NoticeTemplate noticeTemplate, GroupAlert alert) {
String instance = null;
String priority = null;
String content = null;
if (alert.getCommonLabels() != null) {
instance = alert.getCommonLabels().get("instance") == null ? alert.getGroupKey()
: alert.getCommonLabels().get("instance");
priority = alert.getCommonLabels().get("priority") == null ? "unknown"
: alert.getCommonLabels().get("priority");
content = alert.getCommonAnnotations().get("summary");
content = content == null ? alert.getCommonAnnotations().get("description") : content;
if (content == null) {
content = alert.getCommonAnnotations().values().stream().findFirst().orElse(null);
}
}
this.send(receiver.getPhone(), createMessage(instance, priority, content));
}
private String createMessage(String instance, String priority, String content) {
return String.format(MESSAGE_TEMPLATE, instance, priority, content);
}
private void send(String phoneNumber, String message) {
try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
String endpoint = String.format(API_URL_FORMAT, accountSid);
URI requestUri = new URI(endpoint);
HttpPost httpPost = createHttpPost(requestUri, phoneNumber, message);
log.info("Sending Twilio SMS request to {}", requestUri);
executeRequest(httpClient, httpPost, phoneNumber);
} catch (Exception e) {
log.warn("Failed to send SMS: {}", e.getMessage());
throw new SendMessageException(e.getMessage());
}
}
private HttpPost createHttpPost(URI requestUri, String toNumber, String message) {
HttpPost httpPost = new HttpPost(requestUri);
String auth = accountSid + ":" + authToken;
String encodedAuth = Base64.getEncoder().encodeToString(auth.getBytes());
httpPost.setHeader("Authorization", "Basic " + encodedAuth);
List<BasicNameValuePair> parameters = new ArrayList<>();
parameters.add(new BasicNameValuePair("To", toNumber));
parameters.add(new BasicNameValuePair("From", twilioPhoneNumber));
parameters.add(new BasicNameValuePair("Body", message));
try {
httpPost.setEntity(new UrlEncodedFormEntity(parameters));
return httpPost;
} catch (Exception e) {
log.error("Failed to create HTTP request: {}", e.getMessage());
throw new SendMessageException(e.getMessage());
}
}
private void executeRequest(CloseableHttpClient httpClient, HttpPost httpPost, String phoneNumber)
throws Exception {
try (CloseableHttpResponse response = httpClient.execute(httpPost)) {
int statusCode = response.getStatusLine().getStatusCode();
String responseBody = EntityUtils.toString(response.getEntity());
log.info("SMS response status: {}, body: {}", statusCode, responseBody);
if (statusCode < 200 || statusCode >= 300) {
if (responseBody.contains("21608")) {
throw new SendMessageException(
"The Twilio trial account can only send SMS to verified phone numbers");
} else {
throw new SendMessageException(
"HTTP request failed with status code: " + statusCode + ", response: " + responseBody);
}
}
JsonNode jsonResponse = JsonUtil.fromJson(responseBody);
if (jsonResponse == null) {
throw new SendMessageException(statusCode + ":" + responseBody);
}
JsonNode sidNode = jsonResponse.get("sid");
if (sidNode == null) {
throw new SendMessageException(statusCode + ":" + responseBody);
}
String sid = sidNode.asText();
log.info("Successfully sent SMS to phone: {}, sid: {}", phoneNumber, sid);
}
}
@Override
public String getType() {
return TWILIO;
}
@Override
public boolean checkConfig() {
return !(accountSid.isBlank() || authToken.isBlank() || twilioPhoneNumber.isBlank());
}
}
@@ -1,133 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.alert.controller;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.doNothing;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.put;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import static org.springframework.test.web.servlet.setup.MockMvcBuilders.standaloneSetup;
import org.apache.hertzbeat.alert.service.AlertInhibitService;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.alerter.AlertInhibit;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import java.util.HashMap;
/**
* test case for {@link AlertInhibitControllerTest}
*/
@ExtendWith(MockitoExtension.class)
public class AlertInhibitControllerTest {
private MockMvc mockMvc;
@Mock
private AlertInhibitService alertInhibitService;
@InjectMocks
private AlertInhibitController alertInhibitController;
private AlertInhibit alertInhibit;
@BeforeEach
void setUp() {
this.mockMvc = standaloneSetup(alertInhibitController).build();
HashMap<String, String> sourceLabels = new HashMap<>();
HashMap<String, String> targetLabels = new HashMap<>();
alertInhibit = AlertInhibit.builder()
.id(1L)
.name("test")
.sourceLabels(sourceLabels)
.targetLabels(targetLabels)
.creator("test")
.build();
}
@Test
void testAddNewAlertInhibit() throws Exception {
doNothing().when(alertInhibitService).validate(any(AlertInhibit.class), eq(false));
doNothing().when(alertInhibitService).addAlertInhibit(any(AlertInhibit.class));
mockMvc.perform(post("/api/alert/inhibit")
.contentType(MediaType.APPLICATION_JSON)
.content(JsonUtil.toJson(alertInhibit)))
.andExpect(status().isOk())
.andExpect(jsonPath("$.code").value((int) CommonConstants.SUCCESS_CODE))
.andExpect(jsonPath("$.msg").value("Add success"));
}
@Test
void testModifyAlertInhibit() throws Exception {
doNothing().when(alertInhibitService).validate(any(AlertInhibit.class), eq(true));
doNothing().when(alertInhibitService).modifyAlertInhibit(any(AlertInhibit.class));
mockMvc.perform(put("/api/alert/inhibit")
.contentType(MediaType.APPLICATION_JSON)
.content(JsonUtil.toJson(alertInhibit)))
.andExpect(status().isOk())
.andExpect(jsonPath("$.code").value((int) CommonConstants.SUCCESS_CODE))
.andExpect(jsonPath("$.msg").value("Modify success"));
}
@Test
void testGetAlertInhibitExists() throws Exception {
when(alertInhibitService.getAlertInhibit(1L)).thenReturn(alertInhibit);
mockMvc.perform(get("/api/alert/inhibit/{id}", 1L)
.accept(MediaType.APPLICATION_JSON))
.andExpect(status().isOk())
.andExpect(jsonPath("$.data.id").value(alertInhibit.getId()));
}
@Test
void testGetAlertInhibitNotExists() throws Exception {
when(alertInhibitService.getAlertInhibit(1L)).thenReturn(null);
mockMvc.perform(get("/api/alert/inhibit/{id}", 1L)
.accept(MediaType.APPLICATION_JSON))
.andExpect(status().isOk())
.andExpect(jsonPath("$.code").value((int) CommonConstants.MONITOR_NOT_EXIST_CODE))
.andExpect(jsonPath("$.msg").value("AlertInhibit not exist."));
}
}
@@ -41,12 +41,12 @@ import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import java.util.HashMap;
import org.apache.hertzbeat.alert.service.impl.DataSourceServiceImpl;
import org.apache.hertzbeat.warehouse.db.QueryExecutor;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import java.util.List;
import java.util.Map;
import org.mockito.Mockito;
import org.apache.hertzbeat.warehouse.db.QueryExecutor;
/**
* test case for {@link DataSourceService}
@@ -1,64 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.alert.service;
import org.apache.hertzbeat.alert.config.TwilioSmsProperties;
import org.apache.hertzbeat.alert.service.impl.TwilioSmsClientImpl;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.junit.jupiter.api.Assertions.assertEquals;
/**
* Test case for {@link TwilioSmsClientImpl}
*/
class TwilioSmsClientImplTest {
@Test
void getType() {
TwilioSmsProperties twilioSmsProperties = new TwilioSmsProperties();
twilioSmsProperties.setAccountSid("accountSid");
twilioSmsProperties.setAuthToken("authToken");
twilioSmsProperties.setTwilioPhoneNumber("twilioPhoneNumber");
TwilioSmsClientImpl twilioSmsClient = new TwilioSmsClientImpl(twilioSmsProperties);
assertEquals("twilio", twilioSmsClient.getType());
}
@ParameterizedTest
@CsvSource({
"accountSid, authToken, twilioPhoneNumber, true",
"accountSid, authToken, '', false",
"accountSid, '', twilioPhoneNumber, false",
"accountSid, '', '', false",
"'', authToken, twilioPhoneNumber, false",
"'', authToken, '', false",
"'', '', twilioPhoneNumber, false",
"'', '', '', false",
})
void checkConfig(String accountSid, String authToken, String twilioPhoneNumber, boolean expected) {
TwilioSmsProperties twilioSmsProperties = new TwilioSmsProperties();
twilioSmsProperties.setAccountSid(accountSid);
twilioSmsProperties.setAuthToken(authToken);
twilioSmsProperties.setTwilioPhoneNumber(twilioPhoneNumber);
TwilioSmsClientImpl twilioSmsClient = new TwilioSmsClientImpl(twilioSmsProperties);
assertEquals(expected, twilioSmsClient.checkConfig());
}
}
@@ -155,11 +155,6 @@
<artifactId>plc4j-driver-modbus</artifactId>
<version>0.12.0</version>
</dependency>
<dependency>
<groupId>org.apache.plc4x</groupId>
<artifactId>plc4j-driver-s7</artifactId>
<version>0.12.0</version>
</dependency>
<dependency>
<groupId>org.apache.sshd</groupId>
<artifactId>sshd-sftp</artifactId>
@@ -40,10 +40,6 @@ import java.util.stream.Stream;
import javax.net.ssl.SSLException;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpressionException;
import javax.xml.xpath.XPathFactory;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.collector.collect.AbstractCollect;
import org.apache.hertzbeat.collector.collect.common.http.CommonHttpClient;
@@ -67,9 +63,11 @@ import org.apache.hertzbeat.common.util.CommonUtil;
import org.apache.hertzbeat.common.util.IpDomainUtil;
import org.apache.http.Header;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpHost;
import org.apache.http.HttpStatus;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.AuthCache;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.config.RequestConfig;
@@ -79,6 +77,8 @@ import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.client.methods.RequestBuilder;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.auth.DigestScheme;
import org.apache.http.impl.client.BasicAuthCache;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.protocol.HttpContext;
import org.apache.http.util.EntityUtils;
@@ -86,17 +86,10 @@ import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.util.CollectionUtils;
import org.springframework.util.StringUtils;
import org.springframework.web.util.UriUtils;
import org.xml.sax.InputSource;
import org.w3c.dom.Document;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import java.util.Properties;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.HashSet;
import java.util.ArrayList;
import java.util.Collections;
/**
* http https collect
@@ -127,7 +120,7 @@ public class HttpCollectImpl extends AbstractCollect {
if (CollectionUtils.isEmpty(httpProtocol.getSuccessCodes())) {
httpProtocol.setSuccessCodes(List.of(HttpStatus.SC_OK + ""));
}
HttpContext httpContext = createHttpContext(metrics.getHttp());
HttpUriRequest request = createHttpRequest(metrics.getHttp());
try (CloseableHttpResponse response = CommonHttpClient.getHttpClient().execute(request, httpContext)) {
@@ -158,15 +151,13 @@ public class HttpCollectImpl extends AbstractCollect {
case DispatchConstants.PARSE_PROMETHEUS ->
parseResponseByPrometheusExporter(resp, metrics.getAliasFields(), builder);
case DispatchConstants.PARSE_XML_PATH ->
parseResponseByXmlPath(resp, metrics, builder, responseTime);
parseResponseByXmlPath(resp, metrics.getAliasFields(), metrics.getHttp(), builder);
case DispatchConstants.PARSE_WEBSITE ->
parseResponseByWebsite(resp, metrics, metrics.getHttp(), builder, responseTime);
case DispatchConstants.PARSE_SITE_MAP ->
parseResponseBySiteMap(resp, metrics.getAliasFields(), builder);
case DispatchConstants.PARSE_HEADER ->
parseResponseByHeader(builder, metrics.getAliasFields(), response);
case DispatchConstants.PARSE_CONFIG ->
parseResponseByConfig(resp, metrics.getAliasFields(), metrics.getHttp(), builder, responseTime);
default ->
parseResponseByDefault(resp, metrics.getAliasFields(), metrics.getHttp(), builder, responseTime);
}
@@ -337,207 +328,8 @@ public class HttpCollectImpl extends AbstractCollect {
}
}
private void parseResponseByXmlPath(String resp, Metrics metrics,
CollectRep.MetricsData.Builder builder, Long responseTime) {
HttpProtocol http = metrics.getHttp();
List<String> aliasFields = metrics.getAliasFields();
String xpathExpression = http.getParseScript();
if (!StringUtils.hasText(xpathExpression)) {
log.warn("Http collect parse type is xmlPath, but the xpath expression is empty.");
builder.setCode(CollectRep.Code.FAIL);
builder.setMsg("XPath expression is empty");
return;
}
int keywordNum = CollectUtil.countMatchKeyword(resp, http.getKeyword());
try {
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
dbf.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
dbf.setFeature("http://xml.org/sax/features/external-general-entities", false);
dbf.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
dbf.setXIncludeAware(false);
dbf.setExpandEntityReferences(false);
DocumentBuilder db = dbf.newDocumentBuilder();
Document document = db.parse(new InputSource(new StringReader(resp)));
XPathFactory xpathFactory = XPathFactory.newInstance();
XPath xpath = xpathFactory.newXPath();
NodeList nodeList = (NodeList) xpath.evaluate(xpathExpression, document, XPathConstants.NODESET);
if (nodeList == null || nodeList.getLength() == 0) {
log.debug("XPath expression '{}' returned no nodes.", xpathExpression);
boolean requestedSummaryFields = aliasFields.stream()
.anyMatch(alias -> NetworkConstants.RESPONSE_TIME.equalsIgnoreCase(alias)
|| CollectorConstants.KEYWORD.equalsIgnoreCase(alias));
if (requestedSummaryFields) {
CollectRep.ValueRow.Builder valueRowBuilder = CollectRep.ValueRow.newBuilder();
for (String alias : aliasFields) {
if (NetworkConstants.RESPONSE_TIME.equalsIgnoreCase(alias)) {
valueRowBuilder.addColumn(responseTime.toString());
} else if (CollectorConstants.KEYWORD.equalsIgnoreCase(alias)) {
valueRowBuilder.addColumn(Integer.toString(keywordNum));
} else {
valueRowBuilder.addColumn(CommonConstants.NULL_VALUE);
}
}
builder.addValueRow(valueRowBuilder.build());
}
return;
}
for (int i = 0; i < nodeList.getLength(); i++) {
Node node = nodeList.item(i);
CollectRep.ValueRow.Builder valueRowBuilder = CollectRep.ValueRow.newBuilder();
for (String alias : aliasFields) {
if (NetworkConstants.RESPONSE_TIME.equalsIgnoreCase(alias)) {
valueRowBuilder.addColumn(responseTime.toString());
} else if (CollectorConstants.KEYWORD.equalsIgnoreCase(alias)) {
valueRowBuilder.addColumn(Integer.toString(keywordNum));
} else {
try {
String value = (String) xpath.evaluate(alias, node, XPathConstants.STRING);
valueRowBuilder.addColumn(StringUtils.hasText(value) ? value : CommonConstants.NULL_VALUE);
} catch (XPathExpressionException e) {
log.warn("Failed to evaluate XPath '{}' for node [{}]: {}", alias, node.getNodeName(), e.getMessage());
valueRowBuilder.addColumn(CommonConstants.NULL_VALUE);
}
}
}
builder.addValueRow(valueRowBuilder.build());
}
} catch (Exception e) {
log.warn("Failed to parse XML response with XPath '{}': {}", xpathExpression, e.getMessage(), e);
builder.setCode(CollectRep.Code.FAIL);
builder.setMsg("Failed to parse XML response: " + e.getMessage());
}
}
/**
* Parses the response body in Properties/Config format.
* Two modes are supported:
* 1. single-object mode: if http.parseScript is null, aliasFields are treated as indicator names.
* - If there is a locator in the indicator definition, use the locator as the key of the Properties.
* - Otherwise, use aliasField (metric name) as the key for Properties.
* Generate a single row of data.
* 2. array mode: if http.parseScript is not empty (e.g. “users”), treat it as an array base path.
* Treat aliasFields as the attribute name of an array element, and generate a single row of data for each array index. locator is invalid in this mode.
*
* @param resp Response body string
* @param aliasFields List of metrics aliases (i.e., the list of fields in metrics.fields).
* @param http http protocol configuration
* @param builder The metrics data builder.
* @param responseTime response time
*/
private void parseResponseByConfig(String resp, List<String> aliasFields, HttpProtocol http,
CollectRep.MetricsData.Builder builder, Long responseTime) {
if (!StringUtils.hasText(resp)) {
log.warn("Http collect parse type is config, but response body is empty.");
builder.setCode(CollectRep.Code.FAIL);
builder.setMsg("Response body is empty");
return;
}
Properties properties = new Properties();
try (StringReader reader = new StringReader(resp)) {
properties.load(reader);
} catch (IOException e) {
log.warn("Failed to parse config response: {}", e.getMessage(), e);
builder.setCode(CollectRep.Code.FAIL);
builder.setMsg("Failed to parse config response: " + e.getMessage());
return;
}
String arrayBasePath = http.getParseScript();
int keywordNum = CollectUtil.countMatchKeyword(resp, http.getKeyword());
if (!StringUtils.hasText(arrayBasePath)) {
CollectRep.ValueRow.Builder valueRowBuilder = CollectRep.ValueRow.newBuilder();
for (String alias : aliasFields) {
if (NetworkConstants.RESPONSE_TIME.equalsIgnoreCase(alias)) {
valueRowBuilder.addColumn(responseTime.toString());
} else if (CollectorConstants.KEYWORD.equalsIgnoreCase(alias)) {
valueRowBuilder.addColumn(Integer.toString(keywordNum));
} else {
String value = properties.getProperty(alias);
valueRowBuilder.addColumn(value != null ? value : CommonConstants.NULL_VALUE);
}
}
CollectRep.ValueRow valueRow = valueRowBuilder.build();
if (hasMeaningfulDataInRow(valueRow, aliasFields)) {
builder.addValueRow(valueRow);
} else {
log.warn("No meaningful data found in single config object response for aliasFields: {}", aliasFields);
}
} else {
Pattern pattern = Pattern.compile("^" + Pattern.quote(arrayBasePath) + "\\[(\\d+)]\\.");
Set<Integer> existingIndices = new HashSet<>();
for (String key : properties.stringPropertyNames()) {
Matcher matcher = pattern.matcher(key);
if (matcher.find()) {
try {
int index = Integer.parseInt(matcher.group(1));
existingIndices.add(index);
} catch (NumberFormatException e) {
log.error("Could not parse index from key: {}", key);
}
}
}
if (existingIndices.isEmpty()) {
log.warn("Could not find any array elements for base path '{}' in config response.", arrayBasePath);
return;
}
List<Integer> sortedIndices = new ArrayList<>(existingIndices);
Collections.sort(sortedIndices);
for (int i : sortedIndices) {
CollectRep.ValueRow.Builder valueRowBuilder = CollectRep.ValueRow.newBuilder();
for (String alias : aliasFields) {
if (NetworkConstants.RESPONSE_TIME.equalsIgnoreCase(alias)) {
valueRowBuilder.addColumn(responseTime.toString());
} else if (CollectorConstants.KEYWORD.equalsIgnoreCase(alias)) {
valueRowBuilder.addColumn(Integer.toString(keywordNum));
} else {
String currentKey = arrayBasePath + "[" + i + "]." + alias;
String value = properties.getProperty(currentKey);
valueRowBuilder.addColumn(value != null ? value : CommonConstants.NULL_VALUE);
}
}
CollectRep.ValueRow valueRow = valueRowBuilder.build();
if (hasMeaningfulDataInRow(valueRow, aliasFields)) {
builder.addValueRow(valueRowBuilder.build());
}
}
}
}
private boolean hasMeaningfulDataInRow(CollectRep.ValueRow valueRow, List<String> aliasFields) {
if (valueRow.getColumnsCount() == 0) {
return false;
}
if (valueRow.getColumnsCount() != aliasFields.size()) {
log.error("Column count ({}) mismatch with aliasFields size ({}) when checking meaningful data.",
valueRow.getColumnsCount(), aliasFields.size());
return false;
}
boolean hasMeaningfulData = false;
for (int i = 0; i < valueRow.getColumnsCount(); i++) {
String columnValue = valueRow.getColumns(i);
String alias = aliasFields.get(i);
if (!CommonConstants.NULL_VALUE.equals(columnValue) && (!NetworkConstants.RESPONSE_TIME.equalsIgnoreCase(alias) && !CollectorConstants.KEYWORD.equalsIgnoreCase(alias))) {
hasMeaningfulData = true;
break;
}
if ((NetworkConstants.RESPONSE_TIME.equalsIgnoreCase(alias) || CollectorConstants.KEYWORD.equalsIgnoreCase(alias)) && !CommonConstants.NULL_VALUE.equals(columnValue)) {
hasMeaningfulData = true;
}
}
return hasMeaningfulData;
private void parseResponseByXmlPath(String resp, List<String> aliasFields, HttpProtocol http,
CollectRep.MetricsData.Builder builder) {
}
private void parseResponseByJsonPath(String resp, List<String> aliasFields, HttpProtocol http,
@@ -680,10 +472,11 @@ public class HttpCollectImpl extends AbstractCollect {
CredentialsProvider provider = new BasicCredentialsProvider();
UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(auth.getDigestAuthUsername(),
auth.getDigestAuthPassword());
AuthScope authScope = new AuthScope(httpProtocol.getHost(), Integer.parseInt(httpProtocol.getPort()));
provider.setCredentials(authScope, credentials);
provider.setCredentials(AuthScope.ANY, credentials);
AuthCache authCache = new BasicAuthCache();
authCache.put(new HttpHost(httpProtocol.getHost(), Integer.parseInt(httpProtocol.getPort())), new DigestScheme());
clientContext.setCredentialsProvider(provider);
clientContext.setAuthCache(authCache);
return clientContext;
}
}
@@ -716,37 +509,13 @@ public class HttpCollectImpl extends AbstractCollect {
}
// params
Map<String, String> params = httpProtocol.getParams();
boolean enableUrlEncoding = Boolean.parseBoolean(httpProtocol.getEnableUrlEncoding());
StringBuilder queryParams = new StringBuilder();
if (params != null && !params.isEmpty()) {
for (Map.Entry<String, String> param : params.entrySet()) {
String key = param.getKey();
String value = param.getValue();
if (!StringUtils.hasText(key)) {
continue;
}
if (!queryParams.isEmpty()) {
queryParams.append("&");
}
if (enableUrlEncoding) {
key = UriUtils.encodeQueryParam(key, "UTF-8");
}
queryParams.append(key);
if (StringUtils.hasText(value)) {
String calculatedValue = TimeExpressionUtil.calculate(value);
if (enableUrlEncoding) {
calculatedValue = UriUtils.encodeQueryParam(calculatedValue, "UTF-8");
}
queryParams.append("=").append(calculatedValue);
if (StringUtils.hasText(param.getValue())) {
requestBuilder.addParameter(param.getKey(), TimeExpressionUtil.calculate(param.getValue()));
}
}
}
// The default request header can be overridden if customized
// keep-alive
requestBuilder.addHeader(HttpHeaders.CONNECTION, NetworkConstants.KEEP_ALIVE);
@@ -756,7 +525,8 @@ public class HttpCollectImpl extends AbstractCollect {
if (headers != null && !headers.isEmpty()) {
for (Map.Entry<String, String> header : headers.entrySet()) {
if (StringUtils.hasText(header.getValue())) {
requestBuilder.addHeader(header.getKey(), header.getValue());
requestBuilder.addHeader(CollectUtil.replaceUriSpecialChar(header.getKey()),
CollectUtil.replaceUriSpecialChar(header.getValue()));
}
}
}
@@ -787,32 +557,14 @@ public class HttpCollectImpl extends AbstractCollect {
// if it has payload, would override post params
if (StringUtils.hasLength(httpProtocol.getPayload()) && (HttpMethod.POST.matches(httpMethod) || HttpMethod.PUT.matches(httpMethod))) {
requestBuilder.setEntity(new StringEntity(TimeExpressionUtil.calculate(httpProtocol.getPayload()), StandardCharsets.UTF_8));
requestBuilder.setEntity(new StringEntity(httpProtocol.getPayload(), StandardCharsets.UTF_8));
}
// uri encode
String uri;
if (enableUrlEncoding) {
// if the url contains parameters directly
if (httpProtocol.getUrl().contains("?")) {
String path = httpProtocol.getUrl().substring(0, httpProtocol.getUrl().indexOf("?"));
String query = httpProtocol.getUrl().substring(httpProtocol.getUrl().indexOf("?") + 1);
uri = UriUtils.encodePath(path, "UTF-8") + "?" + UriUtils.encodeQuery(query, "UTF-8");
} else {
uri = UriUtils.encodePath(httpProtocol.getUrl(), "UTF-8");
}
} else {
uri = httpProtocol.getUrl();
}
// append query params
if (!queryParams.isEmpty()) {
uri += (uri.contains("?") ? "&" : "?") + queryParams;
}
String finalUri;
// uri
String uri = CollectUtil.replaceUriSpecialChar(httpProtocol.getUrl());
if (IpDomainUtil.isHasSchema(httpProtocol.getHost())) {
finalUri = httpProtocol.getHost() + ":" + httpProtocol.getPort() + uri;
requestBuilder.setUri(httpProtocol.getHost() + ":" + httpProtocol.getPort() + uri);
} else {
String ipAddressType = IpDomainUtil.checkIpAddressType(httpProtocol.getHost());
String baseUri = NetworkConstants.IPV6.equals(ipAddressType)
@@ -820,19 +572,12 @@ public class HttpCollectImpl extends AbstractCollect {
: String.format("%s:%s%s", httpProtocol.getHost(), httpProtocol.getPort(), uri);
boolean ssl = Boolean.parseBoolean(httpProtocol.getSsl());
if (ssl) {
finalUri = NetworkConstants.HTTPS_HEADER + baseUri;
requestBuilder.setUri(NetworkConstants.HTTPS_HEADER + baseUri);
} else {
finalUri = NetworkConstants.HTTP_HEADER + baseUri;
requestBuilder.setUri(NetworkConstants.HTTP_HEADER + baseUri);
}
}
try {
requestBuilder.setUri(finalUri);
} catch (IllegalArgumentException e) {
log.warn("Invalid URI with illegal characters: {}. User has disabled URL encoding, not applying any encoding.", finalUri);
throw e;
}
// custom timeout
int timeout = CollectUtil.getTimeout(httpProtocol.getTimeout(), 0);
if (timeout > 0) {
@@ -33,7 +33,7 @@ import java.util.stream.Stream;
import javax.net.ssl.SSLException;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.collector.collect.common.http.CommonHttpClient;
import org.apache.hertzbeat.common.entity.dto.MetricFamily;
import org.apache.hertzbeat.collector.collect.prometheus.parser.MetricFamily;
import org.apache.hertzbeat.collector.collect.prometheus.parser.TextParser;
import org.apache.hertzbeat.collector.dispatch.DispatchConstants;
import org.apache.hertzbeat.collector.util.CollectUtil;
@@ -15,7 +15,7 @@
* limitations under the License.
*/
package org.apache.hertzbeat.common.entity.dto;
package org.apache.hertzbeat.collector.collect.prometheus.parser;
import java.util.List;
import lombok.Data;
@@ -15,20 +15,18 @@
* limitations under the License.
*/
package org.apache.hertzbeat.common.util;
package org.apache.hertzbeat.collector.collect.prometheus.parser;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.common.entity.dto.MetricFamily;
import java.io.IOException;
import java.io.InputStream;
import java.math.BigDecimal;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
@@ -38,23 +36,9 @@ import java.util.concurrent.ConcurrentHashMap;
@Slf4j
public class OnlineParser {
private static final Map<Integer, Integer> escapeMap = new HashMap<>();
static {
escapeMap.put((int) 'n', (int) '\n');
escapeMap.put((int) 'b', (int) '\b');
escapeMap.put((int) 't', (int) '\t');
escapeMap.put((int) 'r', (int) '\r');
escapeMap.put((int) 'f', (int) '\f');
escapeMap.put((int) '\'', (int) '\'');
escapeMap.put((int) '\"', (int) '\"');
escapeMap.put((int) '\\', (int) '\\');
}
private static class FormatException extends Exception {
public FormatException() {
}
public FormatException() {}
public FormatException(String message) {
super(message);
@@ -139,65 +123,51 @@ public class OnlineParser {
}
private static int getChar(InputStream inputStream) throws IOException, FormatException {
private static CharChecker parseOneChar(InputStream inputStream) throws IOException {
int i = inputStream.read();
if (i == '\\') {
i = inputStream.read();
if (escapeMap.containsKey(i)) {
return escapeMap.get(i);
} else {
throw new FormatException("Escape character failed.");
}
} else {
return i;
}
}
private static CharChecker parseOneChar(InputStream inputStream) throws IOException, FormatException {
int i = getChar(inputStream);
return new CharChecker(i);
}
private static CharChecker parseOneDouble(InputStream inputStream, StringBuilder stringBuilder) throws IOException, FormatException {
int i = getChar(inputStream);
private static CharChecker parseOneDouble(InputStream inputStream, StringBuilder stringBuilder) throws IOException {
int i = inputStream.read();
while ((i >= '0' && i <= '9') || (i >= 'a' && i <= 'z') || (i >= 'A' && i <= 'Z') || i == '-' || i == '+' || i == 'e' || i == '.') {
stringBuilder.append((char) i);
i = getChar(inputStream);
i = inputStream.read();
}
return new CharChecker(i);
}
private static CharChecker skipOneLong(InputStream inputStream) throws IOException, FormatException {
int i = getChar(inputStream);
private static CharChecker skipOneLong(InputStream inputStream) throws IOException {
int i = inputStream.read();
while (i >= '0' && i <= '9') {
i = getChar(inputStream);
i = inputStream.read();
}
return new CharChecker(i);
}
private static CharChecker parseMetricName(InputStream inputStream, StringBuilder stringBuilder) throws IOException, FormatException {
int i = getChar(inputStream);
private static CharChecker parseMetricName(InputStream inputStream, StringBuilder stringBuilder) throws IOException {
int i = inputStream.read();
while ((i >= 'a' && i <= 'z') || (i >= 'A' && i <= 'Z') || (i >= '0' && i <= '9') || i == '_' || i == ':') {
stringBuilder.append((char) i);
i = getChar(inputStream);
i = inputStream.read();
}
return new CharChecker(i);
}
private static CharChecker parseLabelName(InputStream inputStream, StringBuilder stringBuilder) throws IOException, FormatException {
int i = getChar(inputStream);
private static CharChecker parseLabelName(InputStream inputStream, StringBuilder stringBuilder) throws IOException {
int i = inputStream.read();
while ((i >= 'a' && i <= 'z') || (i >= 'A' && i <= 'Z') || (i >= '0' && i <= '9') || i == '_') {
stringBuilder.append((char) i);
i = getChar(inputStream);
i = inputStream.read();
}
return new CharChecker(i);
}
private static CharChecker parseLabelValue(InputStream inputStream, StringBuilder stringBuilder) throws IOException, FormatException {
int i = getChar(inputStream);
int i = inputStream.read();
while (i != '"' && i != -1) {
if (i == '\\') {
i = getChar(inputStream);
i = inputStream.read();
switch (i) {
case 'n':
stringBuilder.append('\n');
@@ -211,26 +181,27 @@ public class OnlineParser {
default:
throw new FormatException();
}
} else {
}
else {
stringBuilder.append((char) i);
}
i = getChar(inputStream);
i = inputStream.read();
}
return new CharChecker(i);
}
private static CharChecker skipSpaces(InputStream inputStream) throws IOException, FormatException {
int i = getChar(inputStream);
private static CharChecker skipSpaces(InputStream inputStream) throws IOException {
int i = inputStream.read();
while (i == ' ') {
i = getChar(inputStream);
i = inputStream.read();
}
return new CharChecker(i);
}
private static CharChecker skipToLineEnd(InputStream inputStream) throws IOException, FormatException {
int i = getChar(inputStream);
private static CharChecker skipToLineEnd(InputStream inputStream) throws IOException {
int i = inputStream.read();
while (i != '\n' && i != -1) {
i = getChar(inputStream);
i = inputStream.read();
}
return new CharChecker(i);
}
@@ -296,18 +267,18 @@ public class OnlineParser {
metricFamily.setMetricList(new ArrayList<>());
metricFamily.setName(metricName);
metricFamilyMap.put(metricName, metricFamily);
} else {
}
else {
metricFamily = metricFamilyMap.get(metricName);
}
if (i == ' ') {
i = skipSpaces(inputStream).getInt();
}
List<MetricFamily.Label> labelList = new LinkedList<>();
metric.setLabels(labelList);
if (i == '{') {
List<MetricFamily.Label> labelList = new LinkedList<>();
parseLabels(inputStream, stringBuilder, labelList);
metric.setLabels(labelList);
i = skipSpaces(inputStream).getInt();
}
@@ -336,8 +307,8 @@ public class OnlineParser {
public static Map<String, MetricFamily> parseMetrics(InputStream inputStream) throws IOException {
Map<String, MetricFamily> metricFamilyMap = new ConcurrentHashMap<>(10);
int i = inputStream.read();
try {
int i = getChar(inputStream);
while (i != -1) {
if (i == '#' || i == '\n') {
skipToLineEnd(inputStream).maybeEol().maybeEof().noElse();
@@ -346,7 +317,7 @@ public class OnlineParser {
stringBuilder.append((char) i);
parseMetric(inputStream, metricFamilyMap, stringBuilder);
}
i = getChar(inputStream);
i = inputStream.read();
}
} catch (FormatException e) {
log.error("prometheus parser failed because of wrong input format. {}", e.getMessage());
@@ -24,7 +24,6 @@ import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.collector.collect.http.promethus.ParseException;
import org.apache.hertzbeat.common.entity.dto.MetricFamily;
import org.apache.hertzbeat.common.util.StrBuffer;
/**
@@ -1,95 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.collector.collect.s7;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.collector.collect.plc.AbstractPlcCollectImpl;
import org.apache.hertzbeat.collector.dispatch.DispatchConstants;
import org.apache.hertzbeat.common.entity.job.Metrics;
import org.apache.hertzbeat.common.entity.job.protocol.PlcProtocol;
import org.apache.hertzbeat.common.entity.job.protocol.S7Protocol;
import org.apache.hertzbeat.common.entity.message.CollectRep;
import org.apache.plc4x.java.api.PlcConnection;
import org.apache.plc4x.java.api.messages.PlcReadRequest;
import org.springframework.beans.BeanUtils;
import org.springframework.util.StringUtils;
import java.util.List;
/**
* plc collect
*/
@Slf4j
public class S7CollectImpl extends AbstractPlcCollectImpl {
@Override
public void preCheck(Metrics metrics) throws IllegalArgumentException {
S7Protocol s7 = metrics.getS7();
List<String> registerAddressList = s7.getRegisterAddresses();
// check RackId
if (!StringUtils.hasText(s7.getRackId())) {
s7.setRackId("0");
}
// check SlotId
if (!StringUtils.hasText(s7.getSlotId())) {
s7.setSlotId("0");
}
// check controllerType
if (!StringUtils.hasText(s7.getControllerType())) {
s7.setControllerType("S7_1500");
}
if (!StringUtils.hasText(s7.getTimeout())) {
s7.setTimeout("5000");
}
PlcProtocol plc = metrics.getPlc() == null ? new PlcProtocol() : metrics.getPlc();
plc.setRegisterAddresses(registerAddressList);
BeanUtils.copyProperties(s7, plc);
metrics.setPlc(plc);
super.preCheck(metrics);
}
@Override
public void collect(CollectRep.MetricsData.Builder builder, Metrics metrics) {
super.collect(builder, metrics);
}
@Override
public String supportProtocol() {
return DispatchConstants.PROTOCOL_S7;
}
@Override
protected String getConnectionString(Metrics metrics) {
S7Protocol s7Protocol = metrics.getS7();
return "s7://" + s7Protocol.getHost() + ":" + s7Protocol.getPort() + "?remote-rack:" + s7Protocol.getRackId()
+ "&remote-slot:" + s7Protocol.getSlotId() + "&controller-type=" + s7Protocol.getControllerType();
}
@Override
protected PlcReadRequest buildRequest(Metrics metrics, PlcConnection connection) {
S7Protocol s7Protocol = metrics.getS7();
List<String> registerAddressList = s7Protocol.getRegisterAddresses();
// Create a new read request:
PlcReadRequest.Builder requestBuilder = connection.readRequestBuilder();
for (int i = 0; i < registerAddressList.size(); i++) {
String s1 = registerAddressList.get(i);
requestBuilder.addTagAddress(metrics.getS7().getAddressSyntax() + ":" + i, s1);
}
return requestBuilder.build();
}
}
@@ -47,28 +47,31 @@ public class DnsCollectImplTest {
.address("www.google.com")
.timeout("3000")
.port("53")
.tcp("tcp")
.build();
}
@Test
public void testPreCheck() {
CollectRep.MetricsData.Builder builder = CollectRep.MetricsData.newBuilder();
//metrics is null
dnsCollect.collect(builder, null);
assertEquals(CollectRep.Code.FAIL, builder.getCode());
assertThrows(IllegalArgumentException.class, () -> {
Metrics metrics = new Metrics();
metrics.setName("question");
metrics.setDns(dnsProtocol);
dnsCollect.preCheck(metrics);
});
//invalid DnsProtocol
Metrics metrics = new Metrics();
//query class is blank
assertThrows(IllegalArgumentException.class, () -> {
DnsProtocol dns = DnsProtocol.builder().build();
Metrics metrics = new Metrics();
metrics.setDns(dns);
dnsCollect.preCheck(metrics);
});
//validated DnsProtocol
// no exception throws
assertDoesNotThrow(() -> {
metrics.setName("question");
dnsProtocol.setTcp("tcp");
Metrics metrics = new Metrics();
metrics.setDns(dnsProtocol);
dnsCollect.preCheck(metrics);
});
@@ -77,23 +80,28 @@ public class DnsCollectImplTest {
@Test
public void testCollect() {
CollectRep.MetricsData.Builder builder = CollectRep.MetricsData.newBuilder();
Metrics metrics0 = Metrics.builder()
.name("question")
.dns(dnsProtocol)
.aliasFields(Collections.singletonList("section"))
.build();
dnsCollect.collect(builder, metrics0);
assertEquals(CollectRep.Code.SUCCESS, builder.getCode());
long monitorId = 666;
String app = "testDNS";
Metrics metrics = new Metrics();
metrics.setName("question");
metrics.setDns(dnsProtocol);
metrics.setAliasFields(Collections.singletonList("section"));
dnsCollect.collect(builder, metrics);
assertNotNull(builder.getValues(0).getColumns(0));
// dns is null, no exception throws
assertDoesNotThrow(() -> {
dnsCollect.collect(builder, null);
});
// metric name is header
Metrics metrics1 = Metrics.builder()
.name("header")
.dns(dnsProtocol)
.aliasFields(Collections.singletonList("section"))
.build();
dnsCollect.collect(builder, metrics1);
assertEquals(CollectRep.Code.SUCCESS, builder.getCode());
assertDoesNotThrow(() -> {
Metrics metrics1 = new Metrics();
metrics1.setName("header");
metrics1.setDns(dnsProtocol);
metrics1.setAliasFields(Collections.singletonList("section"));
dnsCollect.collect(builder, metrics1);
});
}
@Test
@@ -18,14 +18,7 @@
package org.apache.hertzbeat.collector.collect.http;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertEquals;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.List;
import java.util.ArrayList;
import org.apache.hertzbeat.collector.dispatch.DispatchConstants;
import org.apache.hertzbeat.common.entity.job.Metrics;
import org.apache.hertzbeat.common.entity.job.protocol.HttpProtocol;
import org.apache.hertzbeat.common.entity.message.CollectRep;
@@ -48,7 +41,7 @@ class HttpCollectImplTest {
assertThrows(IllegalArgumentException.class, () -> {
httpCollectImpl.preCheck(null);
});
assertThrows(IllegalArgumentException.class, () -> {
Metrics metrics = Metrics.builder().build();
httpCollectImpl.preCheck(metrics);
@@ -72,92 +65,4 @@ class HttpCollectImplTest {
String protocol = httpCollectImpl.supportProtocol();
assert "http".equals(protocol);
}
@Test
void parseResponseByXmlPath() throws Exception {
// Create a sample XML response
String xmlResponse = """
<?xml version="1.0" encoding="UTF-8"?>
<root>
<server>
<name>Server1</name>
<status>Running</status>
<metrics>
<cpu>75.5</cpu>
<memory>1024</memory>
<disk>500</disk>
</metrics>
</server>
<server>
<name>Server2</name>
<status>Stopped</status>
<metrics>
<cpu>0.0</cpu>
<memory>0</memory>
<disk>500</disk>
</metrics>
</server>
</root>
""";
// Set up HttpProtocol with XML path parsing
HttpProtocol http = HttpProtocol.builder()
.parseType(DispatchConstants.PARSE_XML_PATH)
.parseScript("//server") // XPath to select all server nodes
.build();
// Set up Metrics with fields that have XPath expressions
List<Metrics.Field> fields = new ArrayList<>();
fields.add(Metrics.Field.builder().field("name").build());
fields.add(Metrics.Field.builder().field("status").build());
fields.add(Metrics.Field.builder().field("metrics/cpu").build());
fields.add(Metrics.Field.builder().field("metrics/memory").build());
Metrics metrics = Metrics.builder()
.http(http)
.fields(fields)
.aliasFields(Arrays.asList("name", "status", "metrics/cpu", "metrics/memory"))
.build();
// Create a custom builder that captures added rows
List<CollectRep.ValueRow> capturedRows = new ArrayList<>();
CollectRep.MetricsData.Builder builder = new CollectRep.MetricsData.Builder() {
@Override
public CollectRep.MetricsData.Builder addValueRow(CollectRep.ValueRow valueRow) {
capturedRows.add(valueRow);
return super.addValueRow(valueRow);
}
};
// Use reflection to access the private parseResponseByXmlPath method
Method parseMethod = HttpCollectImpl.class.getDeclaredMethod(
"parseResponseByXmlPath",
String.class,
Metrics.class,
CollectRep.MetricsData.Builder.class,
Long.class);
parseMethod.setAccessible(true);
// Call the method
parseMethod.invoke(httpCollectImpl, xmlResponse, metrics, builder, 100L);
// Verify the results
assertEquals(2, capturedRows.size(), "Should have parsed 2 server nodes");
// Check first server
CollectRep.ValueRow firstRow = capturedRows.get(0);
assertEquals(4, firstRow.getColumnsCount(), "First row should have 4 columns");
assertEquals("Server1", firstRow.getColumns(0), "First server name should be Server1");
assertEquals("Running", firstRow.getColumns(1), "First server status should be Running");
assertEquals("75.5", firstRow.getColumns(2), "First server CPU should be 75.5");
assertEquals("1024", firstRow.getColumns(3), "First server memory should be 1024");
// Check second server
CollectRep.ValueRow secondRow = capturedRows.get(1);
assertEquals(4, secondRow.getColumnsCount(), "Second row should have 4 columns");
assertEquals("Server2", secondRow.getColumns(0), "Second server name should be Server2");
assertEquals("Stopped", secondRow.getColumns(1), "Second server status should be Stopped");
assertEquals("0.0", secondRow.getColumns(2), "Second server CPU should be 0.0");
assertEquals("0", secondRow.getColumns(3), "Second server memory should be 0");
}
}
@@ -17,8 +17,6 @@
package org.apache.hertzbeat.collector.collect.prometheus.parser;
import org.apache.hertzbeat.common.entity.dto.MetricFamily;
import org.apache.hertzbeat.common.util.OnlineParser;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
@@ -302,7 +302,7 @@ public class MetricsCollect implements Runnable, Comparable<MetricsCollect> {
value = String.valueOf(objValue);
}
} catch (Exception e) {
log.warn("[calculates execute warning, use original value.] {}", e.getMessage());
log.info("[calculates execute warning] {}.", e.getMessage());
value = Optional.ofNullable(fieldValueMap.get(expression.getSourceText()))
.map(String::valueOf)
.orElse(null);
@@ -29,5 +29,4 @@ org.apache.hertzbeat.collector.collect.mqtt.MqttCollectImpl
org.apache.hertzbeat.collector.collect.ipmi2.IpmiCollectImpl
org.apache.hertzbeat.collector.collect.kafka.KafkaCollectImpl
org.apache.hertzbeat.collector.collect.sd.HttpSdCollectImpl
org.apache.hertzbeat.collector.collect.modbus.ModbusCollectImpl
org.apache.hertzbeat.collector.collect.s7.S7CollectImpl
org.apache.hertzbeat.collector.collect.modbus.ModbusCollectImpl
@@ -39,7 +39,7 @@ spring:
collector:
info:
version: ${COLLECTOR_VERSION:1.7.0}
version: ${COLLECTOR_VERSION:1.6.1}
ip: ${COLLECTOR_IP:}
dispatch:
entrance:
@@ -202,10 +202,6 @@ public interface DispatchConstants {
* Parsing method prometheus exporter data
*/
String PARSE_PROMETHEUS = "prometheus";
/**
* Parse response body as config/properties format
*/
String PARSE_CONFIG = "config";
/**
* prometheus accept header
*/
@@ -231,9 +227,4 @@ public interface DispatchConstants {
* protocol modbus
*/
String PROTOCOL_MODBUS = "modbus";
/**
* protocol modbus
*/
String PROTOCOL_S7 = "s7";
}
@@ -242,7 +242,6 @@ public class KafkaCollectImpl extends AbstractCollect {
boolean isKafkaCommand = SupportedCommand.isKafkaCommand(command);
if (!isKafkaCommand) {
log.error("Unsupported command: {}", command);
builder.setCode(CollectRep.Code.FAIL);
return;
}
@@ -274,7 +273,7 @@ public class KafkaCollectImpl extends AbstractCollect {
}
}
protected AdminClient getAdminClient(KafkaProtocol kafkaProtocol) {
private AdminClient getAdminClient(KafkaProtocol kafkaProtocol) {
CacheIdentifier kafkaAdminClientIdentifier = CacheIdentifier.builder()
.ip(kafkaProtocol.getHost()).port(kafkaProtocol.getPort())
.build();
@@ -17,24 +17,12 @@
package org.apache.hertzbeat.collector.collect.kafka;
import org.apache.hertzbeat.collector.collect.kafka.constants.SupportedCommand;
import org.apache.hertzbeat.collector.dispatch.DispatchConstants;
import org.apache.hertzbeat.common.entity.job.Metrics;
import org.apache.hertzbeat.common.entity.job.protocol.KafkaProtocol;
import org.apache.hertzbeat.common.entity.message.CollectRep;
import org.apache.kafka.clients.admin.AdminClient;
import org.apache.kafka.clients.admin.ListTopicsOptions;
import org.apache.kafka.clients.admin.ListTopicsResult;
import org.apache.kafka.common.KafkaFuture;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import java.util.HashSet;
import java.util.Set;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
@@ -43,25 +31,12 @@ import static org.junit.jupiter.api.Assertions.assertThrows;
/**
* Test case for {@link KafkaCollectImpl}
*/
@ExtendWith(MockitoExtension.class)
public class KafkaCollectTest {
private static final String HOST = "127.0.0.1";
private static final String PORT = "9092";
private KafkaCollectImpl collect;
@Mock
private AdminClient adminClient;
@BeforeEach
public void setUp() throws Exception {
collect = new KafkaCollectImpl(){
@Override
protected AdminClient getAdminClient(KafkaProtocol protocol) {
return adminClient;
}
};
collect = new KafkaCollectImpl();
}
@Test
@@ -84,51 +59,38 @@ public class KafkaCollectTest {
});
// kafka port is null
assertThrows(IllegalArgumentException.class, () -> {
kafka.setHost(HOST);
kafka.setHost("127.0.0.1");
collect.preCheck(metric);
});
// no exception throw
assertDoesNotThrow(() -> {
kafka.setPort(PORT);
kafka.setPort("9092");
collect.preCheck(metric);
});
}
@Test
void testCollect() throws Exception {
Set<String> topicSet = new HashSet<>();
topicSet.add("test-topic");
ListTopicsResult listTopicsResult = Mockito.mock(ListTopicsResult.class);
KafkaFuture<Set<String>> future = Mockito.mock(KafkaFuture.class);
Mockito.when(adminClient.listTopics(Mockito.any(ListTopicsOptions.class))).thenReturn(listTopicsResult);
Mockito.when(listTopicsResult.names()).thenReturn(future);
Mockito.when(future.get()).thenReturn(topicSet);
KafkaProtocol kafka = KafkaProtocol.builder()
.host(HOST)
.port(PORT)
.build();
Metrics metrics = Metrics.builder()
.kclient(kafka)
.build();
// test if not kafka command
CollectRep.MetricsData.Builder builder = CollectRep.MetricsData.newBuilder();
collect.collect(builder, metrics);
assertEquals(CollectRep.Code.FAIL, builder.getCode());
void collect() {
// metrics is null
assertThrows(NullPointerException.class, () -> {
CollectRep.MetricsData.Builder builder = CollectRep.MetricsData.newBuilder();
collect.collect(builder, null);
});
KafkaProtocol kafka = KafkaProtocol.builder().host("127.0.0.1").port("9092").build();
Metrics metrics = Metrics.builder().kclient(kafka).build();
//test if not kafka command
assertDoesNotThrow(() -> {
CollectRep.MetricsData.Builder builder = CollectRep.MetricsData.newBuilder();
collect.collect(builder, metrics);
});
//test kafka command
kafka.setCommand(SupportedCommand.TOPIC_LIST.getCommand());
builder.setCode(CollectRep.Code.forNumber(6));
collect.collect(builder, metrics);
assertDoesNotThrow(() -> {
CollectRep.MetricsData.Builder builder = CollectRep.MetricsData.newBuilder();
metrics.getKclient().setCommand("topic-list");
collect.collect(builder, metrics);
});
assertEquals(CollectRep.Code.SUCCESS, builder.getCode());
assertEquals(1, builder.getValuesList().size());
assertEquals("test-topic", builder.getValues(0).getColumns(0));
Mockito.verify(adminClient).listTopics(Mockito.any(ListTopicsOptions.class));
}
@Test
@@ -455,19 +455,4 @@ public interface CommonConstants {
* status page incident state resolved
*/
byte STATUS_PAGE_INCIDENT_STATE_RESOLVED = 3;
/**
* status page incident state resolved
*/
byte MONITOR_TYPE_NORMAL = 0;
/**
* status page incident state resolved
*/
byte MONITOR_TYPE_PUSH_AUTO_CREATE = 1;
/**
* status page incident state resolved
*/
byte MONITOR_TYPE_DISCOVERY_AUTO_CREATE = 2;
}
@@ -35,7 +35,4 @@ public interface SmsConstants {
// Aws cloud SMS
String AWS = "aws";
// Twilio SMS
String TWILIO = "twilio";
}
@@ -122,11 +122,6 @@ public class NoticeReceiver {
example = "779294123", accessMode = READ_WRITE)
private String tgUserId;
@Schema(title = "Telegram message thread id: The notification method is valid for Telegram Bot",
description = "TTelegram message thread id: The notification method is valid for Telegram Bot",
example = "779294123", accessMode = READ_WRITE)
private String tgMessageThreadId;
@Schema(title = "DingTalk,FeiShu,WeWork user id: The notification method is valid for DingTalk,FeiShu,WeWork Bot",
description = "DingTalk,FeiShu,WeWork user id: The notification method is valid for DingTalk,FeiShu,WeWork Bot",
example = "779294123", accessMode = READ_WRITE)
@@ -1,82 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.common.entity.dto.query;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.util.List;
import java.util.Map;
/**
* Metric History Range Query Data
*/
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
@Schema(description = "Metric Query Data")
public class MetricQueryData {
@Schema(title = "Metric Schema")
private MetricSchema schema;
@Schema(title = "metrics row values, first is the timestamp-ts", example = "[[29,32,44],[32,34,true]]")
private List<List<Object>> values;
/**
* Metric Schema
*/
@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
public static final class MetricSchema {
@Schema(title = "Metrics Field")
private List<MetricField> fields;
@Schema(title = "Meta Information")
private Map<String, String> meta;
}
/**
* Metric Field
*/
@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
public static final class MetricField {
@Schema(title = "Metric Field Name")
private String name;
@Schema(title = "Field Type: number, string, time, bool")
private String type;
@Schema(title = "Field Unit: %, Mb, Kbps etc.")
private String unit;
@Schema(title = "Whether is a label")
private Boolean label;
}
}
@@ -54,7 +54,6 @@ import org.apache.hertzbeat.common.entity.job.protocol.PushProtocol;
import org.apache.hertzbeat.common.entity.job.protocol.RedfishProtocol;
import org.apache.hertzbeat.common.entity.job.protocol.RedisProtocol;
import org.apache.hertzbeat.common.entity.job.protocol.RocketmqProtocol;
import org.apache.hertzbeat.common.entity.job.protocol.S7Protocol;
import org.apache.hertzbeat.common.entity.job.protocol.ScriptProtocol;
import org.apache.hertzbeat.common.entity.job.protocol.SmtpProtocol;
import org.apache.hertzbeat.common.entity.job.protocol.SnmpProtocol;
@@ -271,10 +270,6 @@ public class Metrics {
* Monitoring configuration information using the public modBus protocol
*/
private ModbusProtocol modbus;
/**
* Monitoring configuration information using the public s7 protocol
*/
private S7Protocol s7;
/**
* collector use - Temporarily store subTask metrics response data
*/
@@ -95,12 +95,6 @@ public class HttpProtocol implements CommonRequestProtocol, Protocol {
*/
private List<String> successCodes;
/**
* Whether to enable URL encoding for the path. Default is true.
* When true, the URL path will be encoded. When false, the URL path will not be encoded.
*/
private String enableUrlEncoding = "true";
/**
* authentication information
*/
@@ -91,9 +91,6 @@ public class Monitor {
@Max(4)
private byte status;
@Schema(title = "Task type 0: Normal, 1: push auto create, 2: discovery auto create")
private byte type;
@Schema(title = "task label", example = "{env:test}", accessMode = READ_WRITE)
@Convert(converter = JsonMapAttributeConverter.class)
@Column(length = 4096)
@@ -445,7 +445,15 @@ public final class CollectRep {
if (value != null) {
// Check byte array size, Arrow buffer size is 32768 bytes
byte[] bytes = value.getBytes(StandardCharsets.UTF_8);
vector.setSafe(rowIndex, bytes);
if (bytes.length > 32700) {
log.warn("Value too large for Arrow buffer ({}), truncating to 32700 bytes. Meta: {}",
bytes.length, JsonUtil.toJson(metadata));
byte[] truncatedBytes = new byte[32700];
System.arraycopy(bytes, 0, truncatedBytes, 0, 32700);
vector.set(rowIndex, truncatedBytes);
} else {
vector.set(rowIndex, bytes);
}
}
}
}
@@ -28,7 +28,6 @@ import org.springframework.context.annotation.ImportRuntimeHints;
import org.springframework.data.jpa.repository.config.EnableJpaAuditing;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.annotation.EnableScheduling;
/**
* start up class.
@@ -42,7 +41,6 @@ import org.springframework.scheduling.annotation.EnableScheduling;
@ConfigurationPropertiesScan(basePackages = {"org.apache.hertzbeat"})
@ImportRuntimeHints(HertzbeatRuntimeHintsRegistrar.class)
@EnableAsync
@EnableScheduling
public class Manager {
public static void main(String[] args) {
SpringApplication.run(Manager.class, args);
@@ -15,32 +15,26 @@
* limitations under the License.
*/
package org.apache.hertzbeat.warehouse.service;
package org.apache.hertzbeat.manager.pojo.dto;
import org.apache.hertzbeat.common.entity.dto.query.MetricQueryData;
import java.util.List;
import jakarta.validation.constraints.NotBlank;
import lombok.Data;
/**
* metrics data query service
* Alibaba Sms Sender configuration dto
*/
public interface MetricsDataQueryService {
@Data
public class SmsAlibabaConfig {
/**
* Query metrics data
* @param queries query expr
* @param time time
* @return data
*/
List<MetricQueryData> query(List<String> queries, String queryType, long time);
@NotBlank(message = "AccessKeyId cannot be empty")
private String accessKeyId;
@NotBlank(message = "AccessKeySecret cannot be empty")
private String accessKeySecret;
/**
* Query metrics data range
* @param queries query expr
* @param start start
* @param end end
* @param step step
* @return data
*/
List<MetricQueryData> queryRange(List<String> queries, String queryType, long start, long end, String step);
@NotBlank(message = "SignName cannot be empty")
private String signName;
@NotBlank(message = "TemplateCode cannot be null")
private String templateCode;
}
@@ -0,0 +1,38 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.manager.pojo.dto;
import jakarta.validation.constraints.NotBlank;
import lombok.Data;
/**
* AWS Sms Sender configuration dto
*/
@Data
public class SmsAwsConfig {
@NotBlank(message = "AccessKeyId cannot be empty")
private String accessKeyId;
@NotBlank(message = "AccessKeySecret cannot be empty")
private String accessKeySecret;
@NotBlank(message = "region cannot be empty")
private String region;
}
@@ -15,7 +15,7 @@
* limitations under the License.
*/
package org.apache.hertzbeat.alert.config;
package org.apache.hertzbeat.manager.pojo.dto;
import jakarta.validation.constraints.NotBlank;
import lombok.AllArgsConstructor;
@@ -23,27 +23,25 @@ import lombok.Data;
import lombok.NoArgsConstructor;
/**
* Twilio SMS configuration properties
* Sms Sender configuration dto
*/
@Data
@AllArgsConstructor
@NoArgsConstructor
public class TwilioSmsProperties {
/**
* Twilio Account SID
*/
@NotBlank(message = "Account SID cannot be empty")
private String accountSid;
@AllArgsConstructor
public class SmsNoticeSender {
@NotBlank(message = "Type cannot be empty")
private String type;
private SmsTencentConfig tencent;
private SmsAlibabaConfig alibaba;
/**
* Twilio Auth Token
*/
@NotBlank(message = "Auth Token cannot be empty")
private String authToken;
private SmsUniSmsConfig unisms;
/**
* Twilio Issued Phone Number
*/
@NotBlank(message = "Twilio Phone Number cannot be empty")
private String twilioPhoneNumber;
}
private SmslocalConfig smslocal;
private SmsAwsConfig aws;
private boolean enable = true;
}
@@ -15,45 +15,34 @@
* limitations under the License.
*/
package org.apache.hertzbeat.common.entity.job.protocol;
package org.apache.hertzbeat.manager.pojo.dto;
import jakarta.validation.constraints.NotBlank;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.util.List;
/**
* Modbus Protocol
* Tencent Sms Sender configuration dto
*/
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class S7Protocol implements Protocol {
@AllArgsConstructor
public class SmsTencentConfig {
@NotBlank(message = "SecretId cannot be empty")
private String secretId;
/**
* IP ADDRESS OR DOMAIN NAME OF THE PEER HOST
*/
private String host;
/**
* Port number
*/
private String port;
@NotBlank(message = "SecretKey cannot be empty")
private String secretKey;
private String driverName;
@NotBlank(message = "SignName cannot be empty")
private String signName;
private String addressSyntax;
@NotBlank(message = "AppId cannot be null")
private String appId;
private String rackId;
@NotBlank(message = "templateId cannot be null")
private String templateId;
private String slotId;
private String controllerType;
private String timeout;
private List<String> registerAddresses;
}
@@ -0,0 +1,62 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.hertzbeat.manager.pojo.dto;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.apache.logging.log4j.core.config.plugins.validation.constraints.NotBlank;
/**
* UniSMS configuration
*/
@Data
@NoArgsConstructor
@AllArgsConstructor
public class SmsUniSmsConfig {
/**
* UniSMS access key id
*/
@NotBlank(message = "accessKeyId cannot be empty")
private String accessKeyId;
/**
* UniSMS access key secret, required for HMAC mode
*/
private String accessKeySecret;
/**
* SMS signature
*/
@NotBlank(message = "signature cannot be null")
private String signature;
/**
* SMS template ID
*/
@NotBlank(message = "templateId cannot be null")
private String templateId;
/**
* Authentication mode: simple or hmac, default is simple
*/
@NotBlank(message = "authMode cannot be null")
private String authMode = "simple";
}
@@ -17,8 +17,22 @@
* under the License.
*/
export class TwilioSmsConfig {
accountSid!: string;
authToken!: string;
twilioPhoneNumber!: string;
package org.apache.hertzbeat.manager.pojo.dto;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* Smslocal SMS configuration
*/
@Data
@NoArgsConstructor
@AllArgsConstructor
public class SmslocalConfig {
/**
* Smslocal api key
*/
private String apiKey;
}
@@ -75,8 +75,7 @@ import org.yaml.snakeyaml.Yaml;
/**
* Monitoring Type Management Implementation
* Temporarily stores the monitoring configuration and parameter configuration in memory,
* and then persists it to a storage system.
* temporarily stores the monitoring configuration and parameter configuration in memory and then stores it in the
*/
@Service
@Order(value = Ordered.HIGHEST_PRECEDENCE)
@@ -531,7 +531,7 @@ public class MonitorServiceImpl implements MonitorService {
MonitorDto monitorDto = new MonitorDto();
List<Param> params = paramDao.findParamsByMonitorId(id);
monitorDto.setParams(params);
if (DispatchConstants.PROTOCOL_PROMETHEUS.equalsIgnoreCase(monitor.getApp()) || monitor.getType() == CommonConstants.MONITOR_TYPE_PUSH_AUTO_CREATE) {
if (DispatchConstants.PROTOCOL_PROMETHEUS.equalsIgnoreCase(monitor.getApp())) {
List<CollectRep.MetricsData> metricsDataList = warehouseService.queryMonitorMetricsData(id);
List<String> metrics = metricsDataList.stream().map(CollectRep.MetricsData::getMetrics).collect(Collectors.toList());
monitorDto.setMetrics(metrics);
@@ -22,10 +22,10 @@ import com.fasterxml.jackson.databind.ObjectMapper;
import java.lang.reflect.Type;
import jakarta.annotation.Resource;
import org.apache.hertzbeat.alert.config.SmsConfig;
import org.apache.hertzbeat.common.constants.GeneralConfigTypeEnum;
import org.apache.hertzbeat.base.dao.GeneralConfigDao;
import org.apache.hertzbeat.common.support.event.SmsConfigChangeEvent;
import org.apache.hertzbeat.manager.pojo.dto.SmsNoticeSender;
import org.springframework.context.ApplicationContext;
import org.springframework.stereotype.Service;
@@ -35,7 +35,7 @@ import org.springframework.stereotype.Service;
*/
@Service
public class SmsGeneralConfigServiceImpl extends AbstractGeneralConfigServiceImpl<SmsConfig> {
public class SmsGeneralConfigServiceImpl extends AbstractGeneralConfigServiceImpl<SmsNoticeSender> {
@Resource
private ApplicationContext applicationContext;
@@ -55,7 +55,7 @@ public class SmsGeneralConfigServiceImpl extends AbstractGeneralConfigServiceImp
* This method is used to handle the sms configuration change event.
*/
@Override
public void handler(SmsConfig smsConfig) {
public void handler(SmsNoticeSender smsNoticeSender) {
applicationContext.publishEvent(new SmsConfigChangeEvent(applicationContext));
}
@@ -69,11 +69,11 @@ public class SmsGeneralConfigServiceImpl extends AbstractGeneralConfigServiceImp
* a TypeReference of NoticeSender type
*/
@Override
public TypeReference<SmsConfig> getTypeReference() {
public TypeReference<SmsNoticeSender> getTypeReference() {
return new TypeReference<>() {
@Override
public Type getType() {
return SmsConfig.class;
return SmsNoticeSender.class;
}
};
}
@@ -72,22 +72,14 @@ warehouse:
rpc-port: 6667
username: root
password: root
# store real-time metrics data, enable only one below
real-time:
redis:
enabled: false
host: 127.0.0.1
port: 6379
password: 123456
memory:
enabled: true
init-size: 16
redis:
enabled: false
# redis mode: single, sentinel, cluster. Default is single
mode: single
# separate each address with comma when using cluster mode, eg: 127.0.0.1:6379,127.0.0.1:6380
address: 127.0.0.1:6379
# enter master name when using sentinel mode
masterName: mymaster
password: 123456
# redis db index, default: DB0
db: 0
scheduler:
server:
@@ -110,4 +102,4 @@ ai:
# api key
api-key:
#At present, only IFLYTEK large model needs to be filled in
api-secret:
api-secret:
@@ -231,10 +231,6 @@ alerter:
access-key-id: YOUR_ACCESS_KEY_ID
access-key-secret: YOUR_ACCESS_KEY_SECRET
region: AWS_REGION_FOR_END_USER_MESSAGING
twilio:
account-sid: YOUR_ACCOUNT_SID
auth-token: YOUR_AUTH_TOKEN
twilio-phone-number: YOUR_TWILIO_PHONE_NUMBER
scheduler:
server:
enabled: true
@@ -17,44 +17,71 @@
-- ensure every sql can rerun without error
-- add repair table sql
REPAIR TABLE HZB_ALERT_DEFINE;
DELIMITER //
CREATE PROCEDURE AddOrDropColumns()
BEGIN
DECLARE col_exists INT;
-- Check and add 'app' column if it does not exist
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='app';
IF col_exists = 0 THEN
ALTER TABLE HZB_ALERT_DEFINE ADD COLUMN app VARCHAR(255);
END IF;
-- Drop 'app' column if it exists
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='app';
IF col_exists = 1 THEN
ALTER TABLE HZB_ALERT_DEFINE DROP COLUMN app;
END IF;
-- Check and add 'metric' column if it does not exist
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='metric';
IF col_exists = 0 THEN
ALTER TABLE HZB_ALERT_DEFINE ADD COLUMN metric VARCHAR(255);
END IF;
-- Drop 'metric' column if it exists
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='metric';
IF col_exists = 1 THEN
ALTER TABLE HZB_ALERT_DEFINE DROP COLUMN metric;
END IF;
-- Check and add 'field' column if it does not exist
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='field';
IF col_exists = 0 THEN
ALTER TABLE HZB_ALERT_DEFINE ADD COLUMN field VARCHAR(255);
END IF;
-- Drop 'field' column if it exists
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='field';
IF col_exists = 1 THEN
ALTER TABLE HZB_ALERT_DEFINE DROP COLUMN field;
END IF;
-- Check and add 'preset' column if it does not exist
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='preset';
IF col_exists = 0 THEN
ALTER TABLE HZB_ALERT_DEFINE ADD COLUMN preset BOOLEAN;
END IF;
-- Drop 'preset' column if it exists
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='preset';
IF col_exists = 1 THEN
ALTER TABLE HZB_ALERT_DEFINE DROP COLUMN preset;
END IF;
-- Check and add 'priority' column if it does not exist
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='priority';
IF col_exists = 0 THEN
ALTER TABLE HZB_ALERT_DEFINE ADD COLUMN priority INTEGER;
END IF;
-- Drop 'priority' column if it exists
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='priority';
IF col_exists = 1 THEN
ALTER TABLE HZB_ALERT_DEFINE DROP COLUMN priority;
END IF;
-- Check and add 'tags' column if it does not exist
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='tags';
IF col_exists = 0 THEN
ALTER TABLE HZB_ALERT_DEFINE ADD COLUMN tags VARCHAR(255);
END IF;
-- Drop 'tags' column if it exists
SELECT COUNT(*) INTO col_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='HZB_ALERT_DEFINE' AND COLUMN_NAME='tags';
IF col_exists = 1 THEN
@@ -66,5 +93,4 @@ DELIMITER ;
CALL AddOrDropColumns();
DROP PROCEDURE IF EXISTS AddOrDropColumns;
commit;
DROP PROCEDURE IF EXISTS AddOrDropColumns;
@@ -246,20 +246,6 @@ params:
defaultValue: '200, 201'
# hide param-true or false
hide: true
# field-param field key
- field: enableUrlEncoding
# name-param field display i18n name
name:
zh-CN: 启用URL编码(UTF-8
en-US: Enable URL Encoding(UTF-8)
# type-param field type(most mapping the html input type)
type: boolean
# required-true or false
required: true
# default value
defaultValue: true
# hide param-true or false
hide: true
# collect metrics config list
metrics:
# metrics - summary
@@ -300,8 +286,6 @@ metrics:
method: ^_^httpMethod^_^
# if enabled https
ssl: ^_^ssl^_^
# enable url encoding
enableUrlEncoding: ^_^enableUrlEncoding^_^
# http request payload
payload: ^_^payload^_^
# http request header content
@@ -360,8 +344,6 @@ metrics:
method: GET
# if enabled https
ssl: ^_^ssl^_^
# enable url encoding
enableUrlEncoding: ^_^enableUrlEncoding^_^
authorization:
# http auth type: Basic Auth, Digest Auth, Bearer Token
type: ^_^authType^_^
@@ -1,247 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# The monitoring type categoryservice-application service monitoring db-database monitoring custom-custom monitoring os-operating system monitoring
category: server
# The monitoring type eg: linux windows tomcat mysql aws...
app: dahua
# The monitoring i18n name
name:
zh-CN: 大华
en-US: Dahua
# The description and help of this monitoring type
help:
zh-CN: 通过http接口监控大华设备状态,获取设备健康数据。
en-US: Monitor Dahua devices through http interface to collect health data.
# Input params define for monitoring(render web ui by the definition)
params:
- field: host
name:
zh-CN: 主机Host
en-US: Host
type: host
required: true
- field: port
name:
zh-CN: 端口
en-US: Port
type: number
range: '[0,65535]'
required: true
defaultValue: 80
- field: timeout
name:
zh-CN: 超时时间(ms)
en-US: Timeout(ms)
type: number
range: '[1000,60000]'
required: true
defaultValue: 5000
- field: username
name:
zh-CN: 用户名
en-US: Username
type: text
required: true
- field: password
name:
zh-CN: 密码
en-US: Password
type: password
required: true
- field: ssl
name:
zh-CN: 启用HTTPS
en-US: SSL
type: boolean
required: false
defaultValue: false
# collect metrics config list
metrics:
- name: network_info # 指标集合名称
i18n:
zh-CN: 网络信息
en-US: Network Info
priority: 0
protocol: http
http:
host: ^_^host^_^
port: ^_^port^_^
ssl: ^_^ssl^_^
url: /cgi-bin/configManager.cgi?action=getConfig&name=Network
method: GET
timeout: ^_^timeout^_^
authorization:
type: Digest Auth
digestAuthUsername: ^_^username^_^
digestAuthPassword: ^_^password^_^
parseType: config
fields:
- field: default_interface
type: 1
i18n:
zh-CN: 默认网卡
en-US: Default Interface
- field: domain_name
type: 1
i18n:
zh-CN: 域名
en-US: Domain Name
- field: hostname
type: 1
i18n:
zh-CN: 主机名
en-US: Hostname
- field: eth0_ip_address
type: 1
i18n:
zh-CN: 网卡 eth0 IP地址
en-US: Interface eth0 IP Address
- field: eth0_gateway
type: 1
i18n:
zh-CN: 网卡 eth0 默认网关
en-US: Interface eth0 Default Gateway
- field: eth0_mac_address
type: 1
i18n:
zh-CN: 网卡 eth0 物理地址
en-US: Interface eth0 Physical Address
- field: eth0_subnet_mask
type: 1
i18n:
zh-CN: 网卡 eth0 子网掩码
en-US: Interface eth0 Subnet Mask
- field: eth0_mtu
type: 0
i18n:
zh-CN: 网卡 eth0 MTU
en-US: Interface eth0 MTU
- field: eth0_dns1
type: 1
i18n:
zh-CN: 网卡 eth0 DNS服务器1
en-US: Interface eth0 DNS Server 1
- field: eth0_dns2
type: 1
i18n:
zh-CN: 网卡 eth0 DNS服务器2
en-US: Interface eth0 DNS Server 2
aliasFields:
- table.Network.DefaultInterface
- table.Network.Domain
- table.Network.Hostname
- table.Network.eth0.IPAddress
- table.Network.eth0.DefaultGateway
- table.Network.eth0.PhysicalAddress
- table.Network.eth0.SubnetMask
- table.Network.eth0.MTU
- table.Network.eth0.DnsServers[0]
- table.Network.eth0.DnsServers[1]
calculates:
- default_interface=table.Network.DefaultInterface
- domain_name=table.Network.Domain
- hostname=table.Network.Hostname
- eth0_ip_address=table.Network.eth0.IPAddress
- eth0_gateway=table.Network.eth0.DefaultGateway
- eth0_mac_address=table.Network.eth0.PhysicalAddress
- eth0_subnet_mask=table.Network.eth0.SubnetMask
- eth0_mtu=table.Network.eth0.MTU
- eth0_dns1=table.Network.eth0.DnsServers[0]
- eth0_dns2=table.Network.eth0.DnsServers[1]
- name: user_info
i18n:
zh-CN: 用户信息
en-US: User Info
priority: 0
protocol: http
http:
host: ^_^host^_^
port: ^_^port^_^
ssl: ^_^ssl^_^
url: /cgi-bin/userManager.cgi?action=getActiveUserInfoAll
method: GET
timeout: ^_^timeout^_^
authorization:
type: Digest Auth
digestAuthUsername: ^_^username^_^
digestAuthPassword: ^_^password^_^
parseType: config
parseScript: users
fields:
- field: ClientAddress
type: 1
i18n:
zh-CN: 客户端地址
en-US: ClientAddress
- field: Name
type: 1
i18n:
zh-CN: 客户端用户
en-US: Name
- field: ClientType
type: 1
i18n:
zh-CN: 客户端登录类型
en-US: ClientType
- field: LoginTime
type: 1
i18n:
zh-CN: 客户端登录时间
en-US: LoginTime
- name: ntp_info
i18n:
zh-CN: 校时信息
en-US: Ntp Info
priority: 0
protocol: http
http:
host: ^_^host^_^
port: ^_^port^_^
ssl: ^_^ssl^_^
url: /cgi-bin/configManager.cgi?action=getConfig&name=NTP
method: GET
timeout: ^_^timeout^_^
authorization:
type: Digest Auth
digestAuthUsername: ^_^username^_^
digestAuthPassword: ^_^password^_^
parseType: config
fields:
- field: ntp_address
type: 1
i18n:
zh-CN: 校时服务器
en-US: Ntp Address
- field: ntp_port
type: 1
i18n:
zh-CN: 校时端口
en-US: Ntp Port
- field: ntp_update_period
type: 1
i18n:
zh-CN: 校时间隔
en-US: Ntp Update Period
aliasFields:
- table.NTP.Address
- table.NTP.Port
- table.NTP.UpdatePeriod
calculates:
- ntp_address=table.NTP.Address
- ntp_port=table.NTP.Port
- ntp_update_period=table.NTP.UpdatePeriod
@@ -1,268 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# The monitoring type categoryservice-application service monitoring db-database monitoring custom-custom monitoring os-operating system monitoring
category: server
# The monitoring type eg: linux windows tomcat mysql aws...
app: hikvision_isapi
# The monitoring i18n name
name:
zh-CN: 海康威视 ISAPI
en-US: Hikvision ISAPI
# The description and help of this monitoring type
help:
zh-CN: 通过ISAPI接口监控海康威视设备状态,获取设备健康数据。
en-US: Monitor Hikvision devices through ISAPI interface to collect health data.
# Input params define for monitoring(render web ui by the definition)
params:
- field: host
name:
zh-CN: 主机Host
en-US: Host
type: host
required: true
- field: port
name:
zh-CN: 端口
en-US: Port
type: number
range: '[0,65535]'
required: true
defaultValue: 80
- field: timeout
name:
zh-CN: 超时时间(ms)
en-US: Timeout(ms)
type: number
range: '[1000,60000]'
required: true
defaultValue: 5000
- field: username
name:
zh-CN: 用户名
en-US: Username
type: text
required: true
- field: password
name:
zh-CN: 密码
en-US: Password
type: password
required: true
- field: ssl
name:
zh-CN: 启用HTTPS
en-US: SSL
type: boolean
required: false
defaultValue: false
# collect metrics config list
metrics:
- name: system_info
i18n:
zh-CN: 系统信息
en-US: System Info
priority: 0
protocol: http
http:
host: ^_^host^_^
port: ^_^port^_^
ssl: ^_^ssl^_^
url: /ISAPI/System/deviceInfo
method: GET
timeout: ^_^timeout^_^
authorization:
type: Digest Auth
digestAuthUsername: ^_^username^_^
digestAuthPassword: ^_^password^_^
parseType: xmlPath
parseScript: //DeviceInfo
fields:
- field: deviceName
type: 1
i18n:
zh-CN: 设备名称
en-US: Device Name
- field: deviceID
type: 1
i18n:
zh-CN: 设备ID
en-US: Device ID
- field: firmwareVersion
type: 1
i18n:
zh-CN: 固件版本
en-US: Firmware Version
- field: model
type: 1
i18n:
zh-CN: 设备型号
en-US: Device Model
- field: macAddress
type: 1
i18n:
zh-CN: mac地址
en-US: Mac Address
- name: status
i18n:
zh-CN: 设备状态
en-US: Status
priority: 0
protocol: http
http:
host: ^_^host^_^
port: ^_^port^_^
ssl: ^_^ssl^_^
url: /ISAPI/System/status
method: GET
timeout: ^_^timeout^_^
authorization:
type: Digest Auth
digestAuthUsername: ^_^username^_^
digestAuthPassword: ^_^password^_^
parseType: xmlPath
parseScript: //DeviceStatus
fields:
- field: CPU_utilization
i18n:
zh-CN: CPU 利用率
en-US: CPU Utilization
type: 0
unit: '%'
- field: memory_usage
i18n:
zh-CN: 内存使用量
en-US: Memory Usage
type: 0
unit: MB
- field: memory_available
i18n:
zh-CN: 可用内存
en-US: Memory Available
type: 0
unit: MB
- field: cache_size
i18n:
zh-CN: 缓存大小
en-US: Cache Size
type: 0
unit: MB
- field: net_port_1_speed
i18n:
zh-CN: 网口1速度
en-US: Net Port 1 Speed
type: 0
unit: Mbps
- field: net_port_2_speed
i18n:
zh-CN: 网口2速度
en-US: Net Port 2 Speed
type: 0
unit: Mbps
- field: boot_time
i18n:
zh-CN: 启动时间
en-US: Boot Time
type: 1
- field: device_uptime
i18n:
zh-CN: 运行时长
en-US: Device Uptime
type: 1
- field: last_calibration_time
i18n:
zh-CN: 上次校时时间
en-US: Last Calibration Time
type: 1
- field: last_calibration_time_diff
i18n:
zh-CN: 上次校时时间差
en-US: Last Calibration Time Diff
type: 0
unit: s
- field: avg_upload_time
i18n:
zh-CN: 平均上传耗时
en-US: Avg Upload Time
type: 0
unit: ms
- field: max_upload_time
i18n:
zh-CN: 最大上传耗时
en-US: Max Upload Time
type: 0
unit: ms
- field: min_upload_time
i18n:
zh-CN: 最小上传耗时
en-US: Min Upload Time
type: 0
unit: ms
- field: last_calibration_mode
i18n:
zh-CN: 上次校时模式
en-US: Last Calibration Mode
type: 1
- field: last_calibration_address
i18n:
zh-CN: 上次校时地址
en-US: Last Calibration Address
type: 1
- field: response_time
i18n:
zh-CN: 响应时间
en-US: Response Time
type: 0
unit: ms
aliasFields:
- CPUList/CPU/cpuUtilization
- MemoryList/Memory/memoryUsage
- MemoryList/Memory/memoryAvailable
- MemoryList/Memory/cacheSize
- NetPortStatusList/NetPortStatus[id='1']/workSpeed
- NetPortStatusList/NetPortStatus[id='2']/workSpeed
- bootTime
- deviceUpTime
- lastCalibrationTime
- lastCalibrationTimeDiff
- uploadTimeConsumingList/avgTime
- uploadTimeConsumingList/maxTime
- uploadTimeConsumingList/minTime
- lastCalibrationTimeMode
- lastCalibrationTimeAddress
- responseTime
calculates:
- CPU_utilization=CPUList/CPU/cpuUtilization
- memory_usage=MemoryList/Memory/memoryUsage
- memory_available=MemoryList/Memory/memoryAvailable
- cache_size=MemoryList/Memory/cacheSize
- net_port_1_speed=NetPortStatusList/NetPortStatus[id='1']/workSpeed
- net_port_2_speed=NetPortStatusList/NetPortStatus[id='2']/workSpeed
- boot_time=bootTime
- device_uptime=deviceUpTime
- last_calibration_time=lastCalibrationTime
- last_calibration_time_diff=lastCalibrationTimeDiff
- avg_upload_time=uploadTimeConsumingList/avgTime
- max_upload_time=uploadTimeConsumingList/maxTime
- min_upload_time=uploadTimeConsumingList/minTime
- last_calibration_mode=lastCalibrationTimeMode
- last_calibration_address=lastCalibrationTimeAddress
- response_time=responseTime
units:
- memory_usage=KB->MB
- memory_available=KB->MB
- cache_size=KB->MB
@@ -1,263 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# The monitoring type categoryservice-application service monitoring db-database monitoring mid-middleware custom-custom monitoring os-operating system monitoring
category: service
# The monitoring type eg: linux windows tomcat mysql aws...
app: s7
# The app api i18n name
name:
zh-CN: s7服务器
en-US: s7 Server
# The description and help of this monitoring type
help:
zh-CN: HertzBeat对支持s7协议的服务进行(保持寄存器和线圈)相关指标进行采集
en-US: HertzBeat collects metrics related to maintaining registers and coils for services that support s7 protocol
zh-TW: HertzBeat對支持s7協定的服務進行(保持寄存器和線圈)相關名額進行採集
# Input params define for monitoring(render web ui by the definition)
params:
# field-param field key
- field: host
# name-param field display i18n name
name:
zh-CN: s7服务Host
en-US: s7 Server Host
# type-param field type(most mapping the html input type)
type: host
# required-true or false
required: true
# field-param field key
- field: port
# name-param field display i18n name
name:
zh-CN: 端口
en-US: Port
# type-param field type(most mapping the html input type)
type: number
# when type is number, range is required
range: '[0,65535]'
# required-true or false
required: true
# default value
defaultValue: 102
- field: rackId
# name-param field display i18n name
name:
zh-CN: rackId
en-US: rackId
# type-param field type(most mapping the html input type)
type: text
# required-true or false
required: false
defaultValue: 0
- field: slotId
# name-param field display i18n name
name:
zh-CN: slotId
en-US: slotId
# type-param field type(most mapping the html input type)
type: text
# required-true or false
required: false
defaultValue: 0
- field: controllerType
# name-param field display i18n name
name:
zh-CN: controllerType
en-US: controllerType
# type-param field type(most mapping the html input type)
type: text
# required-true or false
required: false
defaultValue: S7_1500
- field: timeout
name:
zh-CN: 请求超时时间(ms)
en-US: Request Timeout(ms)
type: number
# when type is number, range is required
range: '[400,200000]'
required: false
defaultValue: 6000
hide: true
- field: holdingRegisterAddresses
# name-param field display i18n name
name:
zh-CN: 保持寄存器地址
en-US: Holding Registers address
# type-param field type(most mapping the html input type)
type: array
# param field input placeholder
placeholder: 'Input RegisterAddress'
# required-true or false
required: true
# hide param-true or false
# hide: true
- field: coilRegisterAddresses
# name-param field display i18n name
name:
zh-CN: 线圈寄存器地址
en-US: Coil Register address
# type-param field type(most mapping the html input type)
type: array
# param field input placeholder
placeholder: 'Input RegisterAddress'
# required-true or false
required: true
# hide param-true or false
# hide: true
# collect metrics config list
metrics:
# metrics - summary
- name: holding-register
i18n:
zh-CN: 保持寄存器 统计信息
en-US: holding-register stats
# metrics scheduling priority(0->127)->(high->low), metrics with the same priority will be scheduled in parallel
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 0
# collect metrics content
fields:
- field: responseTime
type: 0
unit: ms
i18n:
zh-CN: 响应时间
en-US: Response Time
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: address-0
type: 0
i18n:
zh-CN: address-0
en-US: address-0
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: address-1
type: 0
i18n:
zh-CN: address-1
en-US: address-1
- field: address-2
type: 0
i18n:
zh-CN: address-2
en-US: address-2
- field: address-3
type: 0
i18n:
zh-CN: address-3
en-US: address-3
# 指标别名列表,按照寄存器地址来进行命名的
# 例如 地址为下标为0 ,值为m holding-register:0
# 地址为下标为1,值为m[2] holding-register:1-0 holding-register:1-1
aliasFields:
- responseTime
- holding-register:0
- holding-register:1
- holding-register:2
- holding-register:3
# mapping and conversion expressions, use thesand aliasField above to calculate metrics value# (可选)指标映射转换计算表达式,与上面的别名一起作用,计算出最终需要的指标值# eg: cores=core1+core2, usage=usage, waitTimeallTime-runningTime
calculates:
- responseTime=responseTime
- address-0=holding-register:0
- address-1=holding-register:1
- address-2=holding-register:2
- address-3=holding-register:3
protocol: s7
# the config content when protocol is http
s7:
# host
host: ^_^host^_^
# port
port: ^_^port^_^
driverName: s7
addressSyntax: holding-register
rackId: ^_^rackId^_^
slotId: ^_^slotId^_^
controllerType: ^_^controllerType^_^
timeout: ^_^timeout^_^
registerAddresses: [ ^_^holdingRegisterAddresses^_^ ]
# metrics - summary
- name: coil
i18n:
zh-CN: 线圈 统计信息
en-US: coil stats
# metrics scheduling priority(0->127)->(high->low), metrics with the same priority will be scheduled in parallel
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 1
# collect metrics content
fields:
- field: responseTime
type: 0
unit: ms
i18n:
zh-CN: 响应时间
en-US: Response Time
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: address-0
type: 0
i18n:
zh-CN: address-0
en-US: address-0
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: address-1
type: 0
i18n:
zh-CN: address-1
en-US: address-1
- field: address-2
type: 0
i18n:
zh-CN: address-2
en-US: address-2
- field: address-3
type: 0
i18n:
zh-CN: address-3
en-US: address-3
# 指标别名列表,按照寄存器地址来进行命名的
# 例如 地址为下标为0 ,值为m coil:0
# 地址为下标为1,值为m[2] coil:1-0 coil:1-1
aliasFields:
- responseTime
- coil:0
- coil:1
- coil:2
- coil:3
# mapping and conversion expressions, use thesand aliasField above to calculate metrics value# (可选)指标映射转换计算表达式,与上面的别名一起作用,计算出最终需要的指标值# eg: cores=core1+core2, usage=usage, waitTimeallTime-runningTime
calculates:
- responseTime=responseTime
- address-0=coil:0
- address-1=coil:1
- address-2=coil:2
- address-3=coil:3
protocol: s7
# the config content when protocol is http
s7:
# host
host: ^_^host^_^
# port
port: ^_^port^_^
driverName: s7
addressSyntax: coil
rackId: ^_^rackId^_^
slotId: ^_^slotId^_^
controllerType: ^_^controllerType^_^
timeout: ^_^timeout^_^
registerAddresses: [ ^_^coilRegisterAddresses^_^ ]
@@ -1,154 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# The monitoring type categoryservice-application service monitoring db-database monitoring custom-custom monitoring os-operating system monitoring
category: server
# The monitoring type eg: linux windows tomcat mysql aws...
app: uniview
# The monitoring i18n name
name:
zh-CN: 宇视
en-US: Uniview
# The description and help of this monitoring type
help:
zh-CN: 通过HTTP接口监控宇视设备状态,获取设备健康数据。
en-US: Monitor Uniview devices through HTTP interface to collect health data.
# Input params define for monitoring(render web ui by the definition)
params:
- field: host
name:
zh-CN: 主机Host
en-US: Host
type: host
required: true
- field: port
name:
zh-CN: 端口
en-US: Port
type: number
range: '[0,65535]'
required: true
defaultValue: 80
- field: timeout
name:
zh-CN: 超时时间(ms)
en-US: Timeout(ms)
type: number
range: '[1000,60000]'
required: true
defaultValue: 5000
- field: username
name:
zh-CN: 用户名
en-US: Username
type: text
required: true
- field: password
name:
zh-CN: 密码
en-US: Password
type: password
required: true
- field: ssl
name:
zh-CN: 启用HTTPS
en-US: SSL
type: boolean
required: false
defaultValue: false
# collect metrics config list
metrics:
- name: system_info
i18n:
zh-CN: 系统信息
en-US: System Info
priority: 0
protocol: http
http:
host: ^_^host^_^
port: ^_^port^_^
ssl: ^_^ssl^_^
url: /LAPI/V1.0/System/DeviceInfo
method: GET
timeout: ^_^timeout^_^
authorization:
type: Digest Auth
digestAuthUsername: ^_^username^_^
digestAuthPassword: ^_^password^_^
parseType: jsonPath
parseScript: '$.Response.Data'
fields:
- field: DeviceName
type: 1
i18n:
zh-CN: 设备名称
en-US: Device Name
- field: SerialNumber
type: 1
i18n:
zh-CN: 序列号
en-US: SerialNumber
- field: FirmwareVersion
type: 1
i18n:
zh-CN: 固件版本
en-US: Firmware Version
- field: DeviceModel
type: 1
i18n:
zh-CN: 设备型号
en-US: Device Model
- name: ntp_info
i18n:
zh-CN: 校时信息
en-US: NTP Info
priority: 0
protocol: http
http:
host: ^_^host^_^
port: ^_^port^_^
ssl: ^_^ssl^_^
url: /LAPI/V1.0/System/Time/NTP
method: GET
timeout: ^_^timeout^_^
authorization:
type: Digest Auth
digestAuthUsername: ^_^username^_^
digestAuthPassword: ^_^password^_^
parseType: jsonPath
parseScript: '$.Response.Data.NTPServerInfos[0]'
fields:
- field: IPAddress
type: 1
i18n:
zh-CN: IP地址
en-US: IPAddress
- field: Port
type: 1
i18n:
zh-CN: 端口号
en-US: Port
- field: SynchronizeInterval
type: 1
i18n:
zh-CN: 同步间隔
en-US: SynchronizeInterval
- field: Enabled
type: 1
i18n:
zh-CN: 是否启用
en-US: Enabled
@@ -21,9 +21,9 @@ import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hertzbeat.alert.config.SmsConfig;
import org.apache.hertzbeat.common.constants.GeneralConfigTypeEnum;
import org.apache.hertzbeat.base.dao.GeneralConfigDao;
import org.apache.hertzbeat.manager.pojo.dto.SmsNoticeSender;
import org.apache.hertzbeat.manager.service.impl.SmsGeneralConfigServiceImpl;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -66,9 +66,9 @@ class SmsGeneralConfigServiceTest {
@Test
void testGetTypeReference() {
TypeReference<SmsConfig> typeReference = service.getTypeReference();
TypeReference<SmsNoticeSender> typeReference = service.getTypeReference();
assertNotNull(typeReference);
assertEquals(SmsConfig.class, typeReference.getType());
assertEquals(SmsNoticeSender.class, typeReference.getType());
}
}
@@ -1,47 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.hertzbeat.push.config;
import org.apache.hertzbeat.push.service.PushGatewayService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.web.servlet.FilterRegistrationBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
*
*/
@Configuration
public class PushFilterConfig {
@Autowired
private PushGatewayService pushGatewayService;
private static final String URI_PREFIX = "/api/push/prometheus/*";
@Bean
public FilterRegistrationBean<PushPrometheusStreamReadingFilter> contentTypeFilter() {
FilterRegistrationBean<PushPrometheusStreamReadingFilter> registrationBean = new FilterRegistrationBean<>();
registrationBean.setFilter(new PushPrometheusStreamReadingFilter(pushGatewayService));
registrationBean.addUrlPatterns(URI_PREFIX);
registrationBean.setOrder(Integer.MIN_VALUE);
return registrationBean;
}
}
@@ -1,82 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.hertzbeat.push.config;
import jakarta.servlet.Filter;
import jakarta.servlet.FilterChain;
import jakarta.servlet.FilterConfig;
import jakarta.servlet.ServletException;
import jakarta.servlet.ServletRequest;
import jakarta.servlet.ServletResponse;
import jakarta.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.apache.hertzbeat.push.service.PushGatewayService;
/**
* todo
*/
public class PushPrometheusStreamReadingFilter implements Filter {
private final PushGatewayService pushGatewayService;
private final Pattern pathPattern = Pattern.compile("^/api/push/prometheus/job/([a-zA-Z0-9_]*)(?:/instance/([a-zA-Z0-9_]*))?$");
public PushPrometheusStreamReadingFilter(PushGatewayService pushGatewayService) {
this.pushGatewayService = pushGatewayService;
}
@Override
public void init(FilterConfig filterConfig) throws ServletException {}
@Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
throws IOException, ServletException {
if (request instanceof HttpServletRequest httpRequest) {
String uri = httpRequest.getRequestURI();
Matcher matcher = pathPattern.matcher(uri);
String job = null;
String instance = null;
if (matcher.matches()) {
job = matcher.group(1);
instance = matcher.group(2);
boolean flag = pushGatewayService.pushPrometheusMetrics(request.getInputStream(), job, instance);
if (flag) {
PushSuccessRequestWrapper successRequestWrapper = new PushSuccessRequestWrapper(httpRequest, job, instance);
chain.doFilter(successRequestWrapper, response);
} else {
PushErrorRequestWrapper errorRequestWrapper = new PushErrorRequestWrapper(httpRequest, job, instance);
chain.doFilter(errorRequestWrapper, response);
}
} else {
chain.doFilter(request, response);
}
} else {
chain.doFilter(request, response);
}
}
@Override
public void destroy() {}
}
@@ -0,0 +1,62 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.push.controller;
import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.tags.Tag;
import org.apache.hertzbeat.common.entity.dto.Message;
import org.apache.hertzbeat.common.entity.push.PushMetricsDto;
import org.apache.hertzbeat.push.service.PushService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
/**
* push controller
*/
@Tag(name = "Metrics Push API")
@RestController
@RequestMapping(value = "/api/push", produces = {APPLICATION_JSON_VALUE})
public class PushController {
@Autowired
private PushService pushService;
@PostMapping
@Operation(summary = "Push metric data to hertzbeat", description = "Push metric data to hertzbeat")
public ResponseEntity<Message<Void>> pushMetrics(@RequestBody PushMetricsDto pushMetricsDto) {
pushService.pushMetricsData(pushMetricsDto);
return ResponseEntity.ok(Message.success("Push success"));
}
@GetMapping()
@Operation(summary = "Get metric data for hertzbeat", description = "Get metric data for hertzbeat")
public ResponseEntity<Message<PushMetricsDto>> getMetrics(
@Parameter(description = "Monitor ID", example = "6565463543") @RequestParam("id") final Long id,
@Parameter(description = "Last pull time", example = "6565463543") @RequestParam("time") final Long time) {
PushMetricsDto pushMetricsDto = pushService.getPushMetricData(id, time);
return ResponseEntity.ok(Message.success(pushMetricsDto));
}
}
@@ -22,9 +22,11 @@ package org.apache.hertzbeat.push.controller;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.io.InputStream;
import org.apache.hertzbeat.common.entity.dto.Message;
import org.apache.hertzbeat.push.config.PushErrorRequestWrapper;
import org.apache.hertzbeat.push.config.PushSuccessRequestWrapper;
import org.apache.hertzbeat.push.service.PushGatewayService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
@@ -35,23 +37,22 @@ import org.springframework.web.bind.annotation.RestController;
*/
@Tag(name = "Metrics Push Gateway API")
@RestController
@RequestMapping(value = "/api/push/prometheus/**")
public class PushPrometheusController {
@RequestMapping(value = "/api/push/pushgateway")
public class PushGatewayController {
@Autowired
private PushGatewayService pushGatewayService;
@PostMapping()
@Operation(summary = "Prometheus push gateway", description = "Push prometheus metric data to hertzbeat")
public ResponseEntity<Message<Void>> pushMetrics(HttpServletRequest request) {
if (request instanceof PushErrorRequestWrapper error) {
return ResponseEntity.badRequest().body(Message.success(String.format("Push failed, job: %s, instance: %s",
error.getJob(), error.getInstance())));
}
else if (request instanceof PushSuccessRequestWrapper success) {
return ResponseEntity.ok(Message.success(String.format("Push success, job: %s, instance: %s",
success.getJob(), success.getInstance())));
@Operation(summary = "Push metric data to hertzbeat pushgateway", description = "Push metric data to hertzbeat pushgateway")
public ResponseEntity<Message<Void>> pushMetrics(HttpServletRequest request) throws IOException {
InputStream inputStream = request.getInputStream();
boolean result = pushGatewayService.pushMetricsData(inputStream);
if (result) {
return ResponseEntity.ok(Message.success("Push success"));
}
else {
return ResponseEntity.badRequest()
.body(Message.success(String.format("Request %s not matched.", request.getRequestURI())));
return ResponseEntity.ok(Message.success("Push failed"));
}
}
@@ -0,0 +1,33 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.push.dao;
import org.apache.hertzbeat.common.entity.push.PushMetrics;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.transaction.annotation.Transactional;
/**
* push metrics dao
*/
public interface PushMetricsDao extends JpaRepository<PushMetrics, Long> {
PushMetrics findFirstByMonitorIdOrderByTimeDesc(Long monitorId);
@Transactional(rollbackFor = Exception.class)
void deleteAllByTimeBefore(Long time);
}
@@ -17,7 +17,6 @@
package org.apache.hertzbeat.push.dao;
import java.util.List;
import org.apache.hertzbeat.common.entity.manager.Monitor;
import org.springframework.data.jpa.repository.JpaRepository;
@@ -25,11 +24,4 @@ import org.springframework.data.jpa.repository.JpaRepository;
* push monitor dao
*/
public interface PushMonitorDao extends JpaRepository<Monitor, Long> {
/**
* Find all monitoring entities by type
* @param type Monitoring type
* @return Monitoring entity list
*/
List<Monitor> findMonitorsByType(byte type);
}
@@ -19,24 +19,17 @@
package org.apache.hertzbeat.push.service;
import java.io.IOException;
import java.io.InputStream;
import org.springframework.stereotype.Service;
/**
* push gateway service
* push gateway metrics
*/
@Service
public interface PushGatewayService {
/**
* push prometheus metrics data
* @param inputStream input stream
* @param job job name, maybe null
* @param instance instance name, maybe null
* @return push success or not
*/
boolean pushPrometheusMetrics(InputStream inputStream, String job, String instance);
boolean pushMetricsData(InputStream inputStream) throws IOException;
}
@@ -0,0 +1,31 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.push.service;
import org.apache.hertzbeat.common.entity.push.PushMetricsDto;
import org.springframework.stereotype.Service;
/**
* push metrics
*/
@Service
public interface PushService {
void pushMetricsData(PushMetricsDto pushMetricsData);
PushMetricsDto getPushMetricData(Long monitorId, Long time);
}
@@ -19,23 +19,9 @@
package org.apache.hertzbeat.push.service.impl;
import java.io.IOException;
import java.io.InputStream;
import java.time.Instant;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.stream.Collectors;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.dto.MetricFamily;
import org.apache.hertzbeat.common.entity.manager.Monitor;
import org.apache.hertzbeat.common.entity.message.CollectRep;
import org.apache.hertzbeat.common.queue.CommonDataQueue;
import org.apache.hertzbeat.common.util.OnlineParser;
import org.apache.hertzbeat.common.util.SnowFlakeIdGenerator;
import org.apache.hertzbeat.push.dao.PushMonitorDao;
import org.apache.hertzbeat.push.service.PushGatewayService;
import org.springframework.stereotype.Service;
@@ -46,88 +32,9 @@ import org.springframework.stereotype.Service;
@Slf4j
@Service
public class PushGatewayServiceImpl implements PushGatewayService {
private final CommonDataQueue commonDataQueue;
private final PushMonitorDao pushMonitorDao;
private final Map<String, Long> jobInstanceMap;
public PushGatewayServiceImpl(CommonDataQueue commonDataQueue, PushMonitorDao pushMonitorDao) {
this.commonDataQueue = commonDataQueue;
this.pushMonitorDao = pushMonitorDao;
jobInstanceMap = new ConcurrentHashMap<>();
pushMonitorDao.findMonitorsByType((byte) 1).forEach(monitor ->
jobInstanceMap.put(monitor.getApp() + "_" + monitor.getName(), monitor.getId()));
}
@Override
public boolean pushPrometheusMetrics(InputStream inputStream, String job, String instance) {
try {
long curTime = Instant.now().toEpochMilli();
Map<String, MetricFamily> metricFamilyMap = OnlineParser.parseMetrics(inputStream);
if (metricFamilyMap == null) {
log.error("parse prometheus metrics is null, job: {}, instance: {}", job, instance);
return false;
}
long id = 0L;
if (job != null && instance != null) {
// auto create monitor when job and instance not null
// job is app, instance is the name
id = jobInstanceMap.computeIfAbsent(job + "_" + instance, key -> {
log.info("auto create monitor by prometheus push, job: {}, instance: {}", job, instance);
long monitorId = SnowFlakeIdGenerator.generateId();
Monitor monitor = Monitor.builder()
.id(monitorId)
.app(job)
.name(instance)
.host(instance)
.type((byte) 1)
.status(CommonConstants.MONITOR_UP_CODE)
.build();
this.pushMonitorDao.save(monitor);
return monitorId;
});
}
for (Map.Entry<String, MetricFamily> entry : metricFamilyMap.entrySet()) {
CollectRep.MetricsData.Builder builder = CollectRep.MetricsData.newBuilder();
builder.setId(id);
builder.setApp(job);
builder.setTime(curTime);
String metricsName = entry.getKey();
builder.setMetrics(metricsName);
MetricFamily metricFamily = entry.getValue();
if (!metricFamily.getMetricList().isEmpty()) {
List<String> metricsFields = new LinkedList<>();
for (int index = 0; index < metricFamily.getMetricList().size(); index++) {
MetricFamily.Metric metric = metricFamily.getMetricList().get(index);
if (index == 0) {
metric.getLabels().forEach(label -> {
metricsFields.add(label.getName());
builder.addField(CollectRep.Field.newBuilder().setName(label.getName())
.setType(CommonConstants.TYPE_STRING).setLabel(true).build());
});
builder.addField(CollectRep.Field.newBuilder().setName("value")
.setType(CommonConstants.TYPE_NUMBER).setLabel(false).build());
}
Map<String, String> labelMap = metric.getLabels()
.stream()
.collect(Collectors.toMap(MetricFamily.Label::getName, MetricFamily.Label::getValue));
CollectRep.ValueRow.Builder valueRowBuilder = CollectRep.ValueRow.newBuilder();
for (String field : metricsFields) {
String fieldValue = labelMap.get(field);
valueRowBuilder.addColumn(fieldValue == null ? CommonConstants.NULL_VALUE : fieldValue);
}
valueRowBuilder.addColumn(String.valueOf(metric.getValue()));
builder.addValueRow(valueRowBuilder.build());
}
commonDataQueue.sendMetricsData(builder.build());
}
}
return true;
} catch (Exception e) {
log.error("push prometheus metrics error", e);
return false;
}
public boolean pushMetricsData(InputStream inputStream) throws IOException {
return true;
}
}
@@ -0,0 +1,143 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.push.service.impl;
import com.fasterxml.jackson.core.type.TypeReference;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Timer;
import java.util.TimerTask;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.common.entity.manager.Monitor;
import org.apache.hertzbeat.common.entity.push.PushMetrics;
import org.apache.hertzbeat.common.entity.push.PushMetricsDto;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.apache.hertzbeat.push.dao.PushMetricsDao;
import org.apache.hertzbeat.push.dao.PushMonitorDao;
import org.apache.hertzbeat.push.service.PushService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
/**
* push service impl
*/
@Slf4j
@Service
public class PushServiceImpl implements PushService {
@Autowired
private PushMonitorDao monitorDao;
@Autowired
private PushMetricsDao metricsDao;
private final Map<Long, Long> monitorIdCache; // key: monitorId, value: time stamp of last query
private static final long cacheTimeout = 5000L; // ms
private final Map<Long, PushMetricsDto.Metrics> lastPushMetrics;
private static final long deleteMetricsPeriod = 1000 * 60 * 60 * 12L;
private static final long deleteBeforeTime = deleteMetricsPeriod / 2;
public PushServiceImpl(){
monitorIdCache = new HashMap<>();
lastPushMetrics = new HashMap<>();
new Timer().schedule(new TimerTask() {
@Override
public void run() {
try {
deletePeriodically();
} catch (Exception e) {
log.error("periodical deletion failed. {}", e.getMessage());
}
}
}, 1000, deleteMetricsPeriod);
}
public void deletePeriodically(){
metricsDao.deleteAllByTimeBefore(System.currentTimeMillis() - deleteBeforeTime);
}
@Override
public void pushMetricsData(PushMetricsDto pushMetricsDto) throws RuntimeException {
List<PushMetrics> pushMetricsList = new ArrayList<>();
long curTime = System.currentTimeMillis();
for (PushMetricsDto.Metrics metrics : pushMetricsDto.getMetricsList()) {
long monitorId = metrics.getMonitorId();
metrics.setTime(curTime);
if (!monitorIdCache.containsKey(monitorId) || (monitorIdCache.containsKey(monitorId) && curTime > monitorIdCache.get(monitorId) + cacheTimeout)) {
Optional<Monitor> queryOption = monitorDao.findById(monitorId);
if (queryOption.isEmpty()) {
monitorIdCache.remove(monitorId);
continue;
}
monitorIdCache.put(monitorId, curTime);
}
PushMetrics pushMetrics = PushMetrics.builder()
.monitorId(metrics.getMonitorId())
.time(curTime)
.metrics(JsonUtil.toJson(metrics.getMetrics())).build();
lastPushMetrics.put(monitorId, metrics);
pushMetricsList.add(pushMetrics);
}
metricsDao.saveAll(pushMetricsList);
}
@Override
public PushMetricsDto getPushMetricData(final Long monitorId, final Long time) {
PushMetricsDto.Metrics metrics;
PushMetricsDto pushMetricsDto = new PushMetricsDto();
if (lastPushMetrics.containsKey(monitorId)) {
metrics = lastPushMetrics.get(monitorId);
}
else {
try {
PushMetrics pushMetrics = metricsDao.findFirstByMonitorIdOrderByTimeDesc(monitorId);
if (pushMetrics == null || pushMetrics.getMetrics() == null) {
return pushMetricsDto;
}
List<Map<String, String>> jsonMap = JsonUtil.fromJson(pushMetrics.getMetrics(), new TypeReference<>() {
});
metrics = PushMetricsDto.Metrics.builder().monitorId(monitorId).metrics(jsonMap).time(pushMetrics.getTime()).build();
lastPushMetrics.put(monitorId, metrics);
}
catch (Exception e) {
log.error("no metrics found, monitor id: {}, {}", monitorId, e.getMessage(), e);
return pushMetricsDto;
}
}
if (time > metrics.getTime()) {
// return void because time param is invalid
return pushMetricsDto;
}
pushMetricsDto.getMetricsList().add(metrics);
return pushMetricsDto;
}
}
@@ -0,0 +1,93 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.hertzbeat.push.controller;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import static org.springframework.test.web.servlet.setup.MockMvcBuilders.standaloneSetup;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.push.PushMetricsDto;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.apache.hertzbeat.push.service.PushService;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
/**
* test case for {@link PushController}
*/
@ExtendWith(MockitoExtension.class)
class PushControllerTest {
private MockMvc mockMvc;
@Mock
private PushService pushService;
@InjectMocks
private PushController pushController;
private PushMetricsDto mockPushMetricsDto;
@BeforeEach
void setUp() {
this.mockMvc = standaloneSetup(this.pushController).build();
mockPushMetricsDto = PushMetricsDto.builder().build();
}
@Test
void testPushMetrics() throws Exception {
this.mockMvc.perform(MockMvcRequestBuilders.post("/api/push")
.contentType(MediaType.APPLICATION_JSON)
.content(JsonUtil.toJson(mockPushMetricsDto)))
.andExpect(status().isOk())
.andExpect(jsonPath("$.code").value((int) CommonConstants.SUCCESS_CODE))
.andReturn();
}
@Test
void testGetMetrics() throws Exception {
Long id = 6565463543L;
Long time = 6565463543L;
when(pushService.getPushMetricData(id, time)).thenReturn(mockPushMetricsDto);
this.mockMvc.perform(MockMvcRequestBuilders.get("/api/push")
.contentType(MediaType.APPLICATION_JSON)
.param("id", id.toString())
.param("time", time.toString()))
.andExpect(status().isOk())
.andExpect(jsonPath("$.code").value((int) CommonConstants.SUCCESS_CODE))
.andReturn();
}
}
@@ -22,11 +22,12 @@ package org.apache.hertzbeat.push.controller;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import java.io.InputStream;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.push.service.PushGatewayService;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
@@ -37,12 +38,11 @@ import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
/**
* test case for {@link PushPrometheusController}
* test case for {@link PushGatewayController}
*/
@Disabled
@ExtendWith(MockitoExtension.class)
class PushPrometheusControllerTest {
class PushGatewayControllerTest {
private MockMvc mockMvc;
@@ -50,7 +50,7 @@ class PushPrometheusControllerTest {
private PushGatewayService pushGatewayService;
@InjectMocks
private PushPrometheusController gatewayController;
private PushGatewayController gatewayController;
@BeforeEach
void setUp() {
@@ -63,12 +63,14 @@ class PushPrometheusControllerTest {
String mockData = "some metric data";
when(pushGatewayService.pushPrometheusMetrics(any(InputStream.class), any(), any())).thenReturn(true);
when(pushGatewayService.pushMetricsData(any(InputStream.class))).thenReturn(true);
mockMvc.perform(post("/api/push/pushgateway")
.contentType(MediaType.APPLICATION_JSON)
.content(mockData))
.andExpect(status().isOk());
.andExpect(status().isOk())
.andExpect(jsonPath("$.code").value((int) CommonConstants.SUCCESS_CODE))
.andExpect(jsonPath("$.msg").value("Push success"));
}
@Test
@@ -76,12 +78,13 @@ class PushPrometheusControllerTest {
String mockData = "some metric data";
when(pushGatewayService.pushPrometheusMetrics(any(InputStream.class), any(), any())).thenReturn(false);
when(pushGatewayService.pushMetricsData(any(InputStream.class))).thenReturn(false);
mockMvc.perform(post("/api/push/pushgateway")
.contentType(MediaType.APPLICATION_JSON)
.content(mockData))
.andExpect(status().isOk());
.andExpect(status().isOk())
.andExpect(jsonPath("$.msg").value("Push failed"));
}
}
@@ -0,0 +1,77 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.push.dao;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.Mockito.doNothing;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import org.apache.hertzbeat.common.entity.push.PushMetrics;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import org.mockito.junit.jupiter.MockitoExtension;
/**
* test case for {@link PushMetricsDao}
*/
@ExtendWith(MockitoExtension.class)
public class PushMetricsDaoTest {
@Mock
private PushMetricsDao pushMetricsDao;
@InjectMocks
private PushMetricsDaoTest pushMetricsDaoTest;
@BeforeEach
void setUp() {
MockitoAnnotations.openMocks(this);
}
@Test
void shallFindFirstByMonitorIdOrderByTimeDesc() {
PushMetrics expectedMetrics = new PushMetrics();
expectedMetrics.setMonitorId(1L);
expectedMetrics.setTime(System.currentTimeMillis());
when(pushMetricsDao.findFirstByMonitorIdOrderByTimeDesc(1L)).thenReturn(expectedMetrics);
PushMetrics actualMetrics = pushMetricsDao.findFirstByMonitorIdOrderByTimeDesc(1L);
assertEquals(expectedMetrics, actualMetrics);
verify(pushMetricsDao, times(1)).findFirstByMonitorIdOrderByTimeDesc(1L);
}
@Test
void shallDeleteAllByTimeBefore() {
doNothing().when(pushMetricsDao).deleteAllByTimeBefore(anyLong());
pushMetricsDao.deleteAllByTimeBefore(1000L);
verify(pushMetricsDao, times(1)).deleteAllByTimeBefore(1000L);
}
}
@@ -0,0 +1,132 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.hertzbeat.push.service;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
import org.apache.hertzbeat.common.entity.manager.Monitor;
import org.apache.hertzbeat.common.entity.push.PushMetrics;
import org.apache.hertzbeat.common.entity.push.PushMetricsDto;
import org.apache.hertzbeat.push.dao.PushMetricsDao;
import org.apache.hertzbeat.push.dao.PushMonitorDao;
import org.apache.hertzbeat.push.service.impl.PushServiceImpl;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.test.util.ReflectionTestUtils;
/**
* test case for {@link PushServiceImpl}
*/
@ExtendWith(MockitoExtension.class)
class PushServiceTest {
@Mock
private PushMonitorDao monitorDao;
@Mock
private PushMetricsDao metricsDao;
@InjectMocks
private PushServiceImpl pushService;
@BeforeEach
void setUp() {
pushService = new PushServiceImpl();
ReflectionTestUtils.setField(pushService, "monitorDao", monitorDao);
ReflectionTestUtils.setField(pushService, "metricsDao", metricsDao);
}
@Test
void testPushMetricsData() {
PushMetricsDto pushMetricsDto = new PushMetricsDto();
List<PushMetricsDto.Metrics> metricsList = new ArrayList<>();
PushMetricsDto.Metrics metrics = new PushMetricsDto.Metrics();
metrics.setMonitorId(1L);
metricsList.add(metrics);
pushMetricsDto.setMetricsList(metricsList);
when(monitorDao.findById(anyLong())).thenReturn(Optional.of(new Monitor()));
pushService.pushMetricsData(pushMetricsDto);
verify(metricsDao, times(1)).saveAll(any());
}
@Test
void testGetPushMetricData() {
Long monitorId = 1L;
Long time = System.currentTimeMillis();
PushMetrics pushMetrics = PushMetrics.builder()
.monitorId(monitorId)
.time(time)
.metrics("[{\"key\":\"value\"}]")
.build();
when(metricsDao.findFirstByMonitorIdOrderByTimeDesc(monitorId)).thenReturn(pushMetrics);
PushMetricsDto result = pushService.getPushMetricData(monitorId, time);
assertEquals(1, result.getMetricsList().size());
assertEquals(monitorId, result.getMetricsList().get(0).getMonitorId());
}
@Test
void testGetPushMetricDataTimeInvalid() {
Long monitorId = 1L;
Long time = System.currentTimeMillis() + 10000;
PushMetrics pushMetrics = PushMetrics.builder()
.monitorId(monitorId)
.time(System.currentTimeMillis())
.metrics("[{\"key\":\"value\"}]")
.build();
when(metricsDao.findFirstByMonitorIdOrderByTimeDesc(monitorId)).thenReturn(pushMetrics);
PushMetricsDto result = pushService.getPushMetricData(monitorId, time);
assertTrue(result.getMetricsList().isEmpty());
}
@Test
void testDeletePeriodically() {
pushService.deletePeriodically();
verify(metricsDao, times(1)).deleteAllByTimeBefore(anyLong());
}
}
@@ -56,8 +56,4 @@ public interface WarehouseConstants {
String MEMORY = "memory";
}
String PROMQL = "promql";
String SQL = "sql";
}
@@ -1,78 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.warehouse.controller;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.tags.Tag;
import org.apache.hertzbeat.common.entity.dto.Message;
import org.apache.hertzbeat.common.entity.dto.query.MetricQueryData;
import org.apache.hertzbeat.warehouse.service.MetricsDataQueryService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import java.util.List;
import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE;
/**
* Metrics Data Query API
*/
@RestController
@RequestMapping(produces = {APPLICATION_JSON_VALUE})
@Tag(name = "Metrics Data Query API")
public class MetricsDataQueryController {
@Autowired
private MetricsDataQueryService queryService;
@GetMapping("/api/warehouse/query")
@Operation(summary = "Query Real Time Metrics Data")
public ResponseEntity<Message<List<MetricQueryData>>> queryMetricsData(
@Parameter(description = "Query PromQL expr list", example = "cpu")
@RequestParam List<String> queries,
@Parameter(description = "Query type", example = "promql")
@RequestParam String type,
@Parameter(description = "Query timestamp", example = "1725854804451")
@RequestParam long time) {
return ResponseEntity.ok(Message.success(queryService.query(queries, type, time)));
}
@GetMapping("/api/warehouse/query/range")
@Operation(summary = "Query Range Metrics Data")
public ResponseEntity<Message<List<MetricQueryData>>> queryMetricsDataRange(
@Parameter(description = "Query PromQL expr list", example = "cpu")
@RequestParam List<String> queries,
@Parameter(description = "Query type", example = "promql")
@RequestParam String type,
@Parameter(description = "Query start timestamp", example = "1725854804451")
@RequestParam long start,
@Parameter(description = "Query end timestamp", example = "1733630804452")
@RequestParam long end,
@Parameter(description = "Query step", example = "4m")
@RequestParam String step
) {
return ResponseEntity.ok(Message.success(queryService.queryRange(queries, type, start, end, step)));
}
}
@@ -18,12 +18,29 @@
package org.apache.hertzbeat.warehouse.db;
import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.common.constants.NetworkConstants;
import org.apache.hertzbeat.common.constants.SignConstants;
import org.apache.hertzbeat.common.util.Base64Util;
import org.apache.hertzbeat.warehouse.store.history.greptime.GreptimeProperties;
import org.apache.hertzbeat.warehouse.store.history.vm.PromQlQueryContent;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Component;
import org.springframework.util.StringUtils;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.util.UriComponentsBuilder;
/**
* query executor for victor metrics
@@ -31,15 +48,71 @@ import org.springframework.web.client.RestTemplate;
@Component
@ConditionalOnProperty(prefix = "warehouse.store.greptime", name = "enabled", havingValue = "true")
@Slf4j
public class GreptimePromqlQueryExecutor extends PromqlQueryExecutor {
public class GreptimePromqlQueryExecutor implements QueryExecutor {
private static final String QUERY_PATH = "/v1/prometheus/api/v1/query";
private final GreptimeProperties greptimeProperties;
private final RestTemplate restTemplate;
public GreptimePromqlQueryExecutor(GreptimeProperties greptimeProperties, RestTemplate restTemplate) {
super(restTemplate, new HttpPromqlProperties(greptimeProperties.httpEndpoint() + QUERY_PATH,
greptimeProperties.username(), greptimeProperties.password()));
this.greptimeProperties = greptimeProperties;
this.restTemplate = restTemplate;
}
@Override
public List<Map<String, Object>> execute(String query) {
// http run the promql query
List<Map<String, Object>> results = new LinkedList<>();
try {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(List.of(MediaType.APPLICATION_JSON));
if (StringUtils.hasText(greptimeProperties.username())
&& StringUtils.hasText(greptimeProperties.password())) {
String authStr = greptimeProperties.username() + ":" + greptimeProperties.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC + SignConstants.BLANK + encodedAuth);
}
HttpEntity<Void> httpEntity = new HttpEntity<>(headers);
URI uri = UriComponentsBuilder.fromHttpUrl(greptimeProperties.httpEndpoint() + QUERY_PATH)
.queryParam("query", URLEncoder.encode(query, StandardCharsets.UTF_8))
.build(true).toUri();
ResponseEntity<PromQlQueryContent> responseEntity = restTemplate.exchange(uri,
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData().getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
Map<String, Object> queryResult = new HashMap<>(8);
queryResult.putAll(labels);
if (content.getValue() != null && content.getValue().length == 2) {
queryResult.put("__timestamp__", content.getValue()[0]);
queryResult.put("__value__", content.getValue()[1]);
} else if (content.getValues() != null && !content.getValues().isEmpty()) {
List<Object> values = new LinkedList<>();
for (Object[] valueArr : content.getValues()) {
values.add(valueArr[1]);
}
queryResult.put("__value__", values);
}
results.add(queryResult);
}
}
} else {
log.error("query metrics data from greptime failed. {}", responseEntity);
}
} catch (Exception e) {
log.error(e.toString(), e);
}
return results;
}
@Override
public boolean support(String datasource) {
return "promql".equals(datasource);
}
}
@@ -1,166 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.hertzbeat.warehouse.db;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.common.constants.NetworkConstants;
import org.apache.hertzbeat.common.constants.SignConstants;
import org.apache.hertzbeat.common.entity.dto.query.MetricQueryData;
import org.apache.hertzbeat.common.util.Base64Util;
import org.apache.hertzbeat.warehouse.store.history.vm.PromQlQueryContent;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpEntity;
import org.springframework.http.MediaType;
import org.springframework.http.HttpMethod;
import org.springframework.http.ResponseEntity;
import org.springframework.util.StringUtils;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.util.UriComponentsBuilder;
import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import static org.apache.hertzbeat.warehouse.constants.WarehouseConstants.PROMQL;
/**
* abstract class for promql query executor
*/
@Slf4j
public abstract class PromqlQueryExecutor implements QueryExecutor {
private static final String supportQueryLanguage = PROMQL;
protected static final String HTTP_QUERY_PARAM = "query";
protected static final String HTTP_TIME_PARAM = "time";
protected static final String HTTP_START_PARAM = "start";
protected static final String HTTP_END_PARAM = "end";
protected static final String HTTP_STEP_PARAM = "step";
private final RestTemplate restTemplate;
private final HttpPromqlProperties httpPromqlProperties;
PromqlQueryExecutor(RestTemplate restTemplate, HttpPromqlProperties httpPromqlProperties) {
this.restTemplate = restTemplate;
this.httpPromqlProperties = httpPromqlProperties;
}
/**
* record class for promql http connection
*/
protected record HttpPromqlProperties (
String url,
String username,
String password
){}
protected List<Map<String, Object>> http_promql(Map<String, Object> params) {
// http run the promql query
List<Map<String, Object>> results = new LinkedList<>();
try {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(List.of(MediaType.APPLICATION_JSON));
if (StringUtils.hasText(httpPromqlProperties.username())
&& StringUtils.hasText(httpPromqlProperties.password())) {
String authStr = httpPromqlProperties.username() + ":" + httpPromqlProperties.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC + SignConstants.BLANK + encodedAuth);
}
HttpEntity<Void> httpEntity = new HttpEntity<>(headers);
UriComponentsBuilder uriComponentsBuilder = UriComponentsBuilder.fromHttpUrl(httpPromqlProperties.url);
for (Map.Entry<String, Object> entry : params.entrySet()) {
uriComponentsBuilder.queryParam(entry.getKey(), entry.getValue());
}
URI uri = uriComponentsBuilder.build(true).toUri();
ResponseEntity<PromQlQueryContent> responseEntity = restTemplate.exchange(uri,
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData().getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
Map<String, Object> queryResult = new HashMap<>(8);
queryResult.putAll(labels);
if (content.getValue() != null && content.getValue().length == 2) {
queryResult.put("__timestamp__", content.getValue()[0]);
queryResult.put("__value__", content.getValue()[1]);
} else if (content.getValues() != null && !content.getValues().isEmpty()) {
List<Object> values = new LinkedList<>();
for (Object[] valueArr : content.getValues()) {
values.add(valueArr[1]);
}
queryResult.put("__value__", values);
}
results.add(queryResult);
}
}
} else {
log.error("query metrics data from greptime failed. {}", responseEntity);
}
} catch (Exception e) {
log.error(e.toString(), e);
}
return results;
}
public MetricQueryData convertToMetricQueryData(Object object) {
MetricQueryData metricQueryData = new MetricQueryData();
try {
List<Map<String, Object>> metrics = (List<Map<String, Object>>) object;
// todo
} catch (Exception e) {
log.error("converting to metric query data failed.");
}
return metricQueryData;
}
public List<Map<String, Object>> execute(String queryString) {
Map<String, Object> params = new HashMap<>();
params.put(HTTP_QUERY_PARAM, URLEncoder.encode(queryString, StandardCharsets.UTF_8));
return http_promql(params);
}
public List<Map<String, Object>> query(String queryString, long time) {
Map<String, Object> params = new HashMap<>();
params.put(HTTP_QUERY_PARAM, URLEncoder.encode(queryString, StandardCharsets.UTF_8));
params.put(HTTP_TIME_PARAM, time);
return http_promql(params);
}
public List<Map<String, Object>> query_range(String queryString, long start, long end, String step) {
Map<String, Object> params = new HashMap<>();
params.put(HTTP_QUERY_PARAM, URLEncoder.encode(queryString, StandardCharsets.UTF_8));
params.put(HTTP_START_PARAM, start);
params.put(HTTP_END_PARAM, end);
params.put(HTTP_STEP_PARAM, step);
return http_promql(params);
}
public boolean support(String datasource) {
return supportQueryLanguage.equals(datasource);
}
}
@@ -17,8 +17,6 @@
package org.apache.hertzbeat.warehouse.db;
import org.apache.hertzbeat.common.entity.dto.query.MetricQueryData;
import java.util.List;
import java.util.Map;
@@ -26,14 +24,8 @@ import java.util.Map;
* query executor interface
*/
public interface QueryExecutor {
MetricQueryData convertToMetricQueryData(Object object);
List<Map<String, Object>> execute(String query);
List<Map<String, Object>> query(String query, long time);
List<Map<String, Object>> query_range(String query, long start, long end, String step);
boolean support(String datasource);
}
@@ -1,67 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.hertzbeat.warehouse.db;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.common.entity.dto.query.MetricQueryData;
import java.util.List;
import java.util.Map;
import static org.apache.hertzbeat.warehouse.constants.WarehouseConstants.SQL;
/**
* abstract class for sql query executor
*/
@Slf4j
public abstract class SqlQueryExecutor implements QueryExecutor {
private static final String supportQueryLanguage = SQL;
/**
* record class for sql connection
*/
protected record ConnectorSqlProperties () {}
protected abstract List<Map<String, Object>> do_sql(Map<String, Object> params);
public MetricQueryData convertToMetricQueryData(Object object) {
MetricQueryData metricQueryData = new MetricQueryData();
try {
List<Map<String, Object>> metrics = (List<Map<String, Object>>) object;
// todo
} catch (Exception e) {
log.error("converting to metric query data failed.");
}
return metricQueryData;
}
public abstract List<Map<String, Object>> execute(String query);
public abstract List<Map<String, Object>> query(String query, long time);
public abstract List<Map<String, Object>> query_range(String query, long start, long end, String step);
public boolean support(String datasource) {
return supportQueryLanguage.equals(datasource);
}
}
@@ -18,11 +18,29 @@
package org.apache.hertzbeat.warehouse.db;
import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.common.constants.NetworkConstants;
import org.apache.hertzbeat.common.constants.SignConstants;
import org.apache.hertzbeat.common.util.Base64Util;
import org.apache.hertzbeat.warehouse.store.history.vm.PromQlQueryContent;
import org.apache.hertzbeat.warehouse.store.history.vm.VictoriaMetricsProperties;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Component;
import org.springframework.util.StringUtils;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.util.UriComponentsBuilder;
/**
* query executor for victor metrics
@@ -30,16 +48,71 @@ import org.springframework.web.client.RestTemplate;
@Component
@ConditionalOnProperty(prefix = "warehouse.store.victoria-metrics", name = "enabled", havingValue = "true")
@Slf4j
public class VictoriaMetricsQueryExecutor extends PromqlQueryExecutor {
public class VictoriaMetricsQueryExecutor implements QueryExecutor {
private static final String QUERY_PATH = "/api/v1/query";
private final VictoriaMetricsProperties victoriaMetricsProp;
private final RestTemplate restTemplate;
public VictoriaMetricsQueryExecutor(VictoriaMetricsProperties victoriaMetricsProp, RestTemplate restTemplate) {
super(restTemplate, new HttpPromqlProperties(victoriaMetricsProp.url() + QUERY_PATH,
victoriaMetricsProp.username(), victoriaMetricsProp.password()));
this.victoriaMetricsProp = victoriaMetricsProp;
this.restTemplate = restTemplate;
}
@Override
public List<Map<String, Object>> execute(String query) {
// http run the promql query
List<Map<String, Object>> results = new LinkedList<>();
try {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(List.of(MediaType.APPLICATION_JSON));
if (StringUtils.hasText(victoriaMetricsProp.username())
&& StringUtils.hasText(victoriaMetricsProp.password())) {
String authStr = victoriaMetricsProp.username() + ":" + victoriaMetricsProp.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC + SignConstants.BLANK + encodedAuth);
}
HttpEntity<Void> httpEntity = new HttpEntity<>(headers);
URI uri = UriComponentsBuilder.fromHttpUrl(victoriaMetricsProp.url() + QUERY_PATH)
.queryParam("query", URLEncoder.encode(query, StandardCharsets.UTF_8))
.build(true).toUri();
ResponseEntity<PromQlQueryContent> responseEntity = restTemplate.exchange(uri,
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData().getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
Map<String, Object> queryResult = new HashMap<>(8);
queryResult.putAll(labels);
if (content.getValue() != null && content.getValue().length == 2) {
queryResult.put("__timestamp__", content.getValue()[0]);
queryResult.put("__value__", content.getValue()[1]);
} else if (content.getValues() != null && !content.getValues().isEmpty()) {
List<Object> values = new LinkedList<>();
for (Object[] valueArr : content.getValues()) {
values.add(valueArr[1]);
}
queryResult.put("__value__", values);
}
results.add(queryResult);
}
}
} else {
log.error("query metrics data from victor-metrics failed. {}", responseEntity);
}
} catch (Exception e) {
log.error(e.toString(), e);
}
return results;
}
@Override
public boolean support(String datasource) {
return "promql".equals(datasource);
}
}
@@ -1,67 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.hertzbeat.warehouse.service.impl;
import org.apache.hertzbeat.common.entity.dto.query.MetricQueryData;
import org.apache.hertzbeat.warehouse.db.QueryExecutor;
import org.apache.hertzbeat.warehouse.service.MetricsDataQueryService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.util.ArrayList;
import java.util.List;
@Service
public class MetricsDataQueryServiceImpl implements MetricsDataQueryService {
@Autowired(required = false)
List<QueryExecutor> executors;
@Override
public List<MetricQueryData> query(List<String> queries, String queryType, long time) {
if (queries == null || executors.isEmpty()) {
throw new IllegalArgumentException("No query executor found");
}
QueryExecutor executor = executors.stream().filter(e -> e.support(queryType)).findFirst().orElse(null);
if (executor == null) {
throw new IllegalArgumentException("Unsupported datasource: " + queryType);
}
List<MetricQueryData> metricQueryDataList = new ArrayList<>();
for (String query : queries) {
metricQueryDataList.add(executor.convertToMetricQueryData(executor.query(query, time)));
}
return metricQueryDataList;
}
@Override
public List<MetricQueryData> queryRange(List<String> queries, String queryType, long start, long end, String step) {
if (queries == null || executors.isEmpty()) {
throw new IllegalArgumentException("No query executor found");
}
QueryExecutor executor = executors.stream().filter(e -> e.support(queryType)).findFirst().orElse(null);
if (executor == null) {
throw new IllegalArgumentException("Unsupported datasource: " + queryType);
}
List<MetricQueryData> metricQueryDataList = new ArrayList<>();
for (String query : queries) {
metricQueryDataList.add(executor.convertToMetricQueryData(executor.query_range(query, start, end, step)));
}
return metricQueryDataList;
}
}
@@ -1,106 +0,0 @@
---
title: "Behind the Scenes of HertzBeat: How Metric Collection Works"
author: JuJinPark
author_title: JuJin
author_url: https://github.com/JuJinPark
tags: [opensource, practice]
keywords: [open source monitoring system]
---
## Behind the Scenes of HertzBeat: How Metric Collection Works
HertzBeat is an open-source, real-time monitoring system designed for flexibility and ease of use. But how exactly does it collect, process, and store metrics from various systems?
In this post, well walk through the internal architecture behind **HertzBeats metric collection pipeline** — from job distribution to alerting and storage — with the help of a high-level system diagram.
---
### HertzBeats Metric Collection Architecture
![HertzBeat Architecture](/img/docs/hertzbeat-metrics-collection-arch.png)
> **Figure:** High-level architecture of HertzBeat's metric collection system. The Manager handles job scheduling, alerting, and storage, while Collectors (external or internal) perform the actual metric collection. Communication between the Manager and Collectors uses a custom Netty TCP protocol.
---
### 1. Job Distribution: Assigning What to Monitor
When the **Manager** component starts, it loads monitoring targets from the database. These targets define the host, collection interval, and other parameters.
To distribute the workload, the Manager sends jobs to **external Collectors** over a custom **Netty-based TCP protocol**. The `CollectJobScheduling` module handles this logic using **consistent hashing**, ensuring jobs are evenly distributed across collectors.
> 💡 HertzBeat also includes a built-in **main collector** (identified as `MAIN_COLLECTOR_NODE`) that runs directly inside the Manager. This allows HertzBeat to operate in **standalone mode** without requiring any external collectors.
---
### 2. Task Scheduling: When to Monitor
Once a Collector receives a job, it registers it with the **`TimerDispatch`** system.
- For **external collectors**, the Manager sends the task via the TCP connection.
- For the **main collector**, the Manager directly invokes `CollectJobService` within the same process.
Each Collector runs a **`Timer`** in a background thread, which schedules tasks according to their configured intervals. When the time is up, the timer triggers a `TimerTask` to begin metric collection.
---
### 3. Task Execution: How Metrics Are Collected
When a `TimerTask` is triggered, it creates a `MetricsCollect` task and passes it to `MetricsTaskDispatch`, which places it in the **`MetricsCollectorQueue`**.
- A dedicated thread (`CommonDispatcher`) continuously polls this queue.
- Tasks are executed by a **worker thread pool**, allowing multiple metric collections to run concurrently.
- Each task uses a specific **collector strategy** (e.g., HTTP, JDBC, SSH) to fetch metrics from the target system.
---
### 4. Result Processing: What Happens to Collected Data
Once metrics are collected, the results are processed by the **`CollectDataDispatch`** module.
- If the task is recurring, it is rescheduled via `TimerDispatch`.
- Results are added to a **`CommonDataQueue`** for further handling.
For external collectors, results are sent **back to the Manager** via the Netty TCP connection. For the main collector, results are forwarded **directly** to the next processing stage without network overhead.
---
### 5. Alerting & Storage: Making Metrics Useful
The Manager receives metric data and pushes it into the `MetricsDataToAlertQueue`, where it is processed through two main pipelines:
#### 🔔 Alerting
- The `RealTimeAlertCalculator` consumes metrics from the alert queue.
- It checks each metric against user-defined alert rules and triggers alerts if conditions are met.
#### 🧠 Storage
- After alert evaluation, metrics are added to the `MetricsDataToStorageQueue`.
- A background thread (`DataStorageDispatch`) processes this queue and stores the metrics in a database for long-term analysis and dashboard visualization.
---
### Standalone Mode: No External Collectors Required
Thanks to the built-in **main collector**, HertzBeat can operate entirely in standalone mode. This is especially useful for testing, small deployments, or quick setup. All core components — job scheduling, collection, alerting, and storage — run within a single process.
---
### 🧠 Conclusion
HertzBeats metric collection system is designed for **performance, scalability, and flexibility**. With its:
- **Queue-based, multi-threaded architecture**
- **Persistent TCP connections** for reliable job/result flow
- **Built-in main collector** for standalone operation
it handles large-scale monitoring workloads with minimal overhead and high efficiency.
---
### 🙌 Whats Next?
If you're curious to explore more:
- ⭐️ [Star the project on GitHub](https://github.com/apache/hertzbeat)
- 🤝 [Contribute or open an issue](https://github.com/apache/hertzbeat/issues)
-326
View File
@@ -1,326 +0,0 @@
---
id: extend-http-xmlpath
title: HTTP Protocol XmlPath Parsing Method
sidebar_label: XmlPath Parsing Method
---
> After calling the HTTP API to obtain the response data, use the XmlPath script parsing method to parse the response data.
Note⚠️ The response data must be in XML format.
**Use XPath scripts to parse the response data into data that conforms to the data structure rules specified by HertzBeat.**
### XmlPath Parsing Logic
The XmlPath parsing method in HertzBeat uses a two-step XPath process:
1. **Main XPath Expression (`parseScript`)**: This XPath expression is defined in the `http` configuration section under `parseScript`. It is used to select one or more main XML nodes from the response. Each selected node will correspond to one row of metric data in HertzBeat.
2. **Relative Field XPath Expressions (`xpath`)**: For each metric field defined in the `fields` list, you can specify a relative `xpath`. This XPath expression is evaluated *relative to each main node* selected by the `parseScript` in step 1. It extracts the specific value for that metric field from the current main node.
This allows you to easily parse structured XML data where multiple records or items are present.
**Special Metrics**:
* `responseTime`: This built-in metric represents the HTTP request's response time and is automatically collected. It does not require an `xpath`.
* `keyword`: This built-in metric counts the occurrences of a specified keyword (configured in `http.keyword`) in the raw response body. It does not require an `xpath`.
### Example
Assume the HTTP API returns the following XML data:
```xml
<DeviceStatus xmlns="http://www.isapi.org/ver20/XMLSchema" version="2.0">
<CPUList>
<CPU>
<cpuUtilization>36.400002</cpuUtilization>
</CPU>
</CPUList>
<MemoryList>
<Memory>
<memoryUsage>399640</memoryUsage>
<memoryAvailable>98792</memoryAvailable>
<cacheSize>228492</cacheSize>
</Memory>
</MemoryList>
<NetPortStatusList>
<NetPortStatus>
<id>1</id>
<workSpeed>1000</workSpeed>
</NetPortStatus>
<NetPortStatus>
<id>2</id>
<workSpeed>0</workSpeed>
</NetPortStatus>
</NetPortStatusList>
<bootTime>2025-01-06 10:27:48</bootTime>
<deviceUpTime>87天0时55分59秒</deviceUpTime>
<lastCalibrationTime>2025-04-03 11:09:18</lastCalibrationTime>
<lastCalibrationTimeDiff>1</lastCalibrationTimeDiff>
<uploadTimeConsumingList>
<avgTime>16</avgTime>
<maxTime>23</maxTime>
<minTime>12</minTime>
</uploadTimeConsumingList>
<lastCalibrationTimeMode>NTP</lastCalibrationTimeMode>
<lastCalibrationTimeAddress>34.191.45.101</lastCalibrationTimeAddress>
</DeviceStatus>
```
We want to monitor the device status and extract various metrics.
Here's how you would configure the monitoring template YML:
```yaml
category: server
# The monitoring type eg: linux windows tomcat mysql aws...
app: hikvision_isapi
# The monitoring i18n name
name:
zh-CN: 海康威视 ISAPI
en-US: Hikvision ISAPI
# The description and help of this monitoring type
help:
zh-CN: 通过ISAPI接口监控海康威视设备状态,获取设备健康数据。
en-US: Monitor Hikvision devices through ISAPI interface to collect health data.
# Input params define for monitoring(render web ui by the definition)
params:
- field: host
name:
zh-CN: 主机Host
en-US: Host
type: host
required: true
- field: port
name:
zh-CN: 端口
en-US: Port
type: number
range: '[0,65535]'
required: true
defaultValue: 80
- field: timeout
name:
zh-CN: 超时时间(ms)
en-US: Timeout(ms)
type: number
range: '[1000,60000]'
required: true
defaultValue: 5000
- field: username
name:
zh-CN: 用户名
en-US: Username
type: text
required: true
- field: password
name:
zh-CN: 密码
en-US: Password
type: password
required: true
- field: ssl
name:
zh-CN: 启用HTTPS
en-US: SSL
type: boolean
required: false
defaultValue: false
# collect metrics config list
metrics:
- name: system_info
i18n:
zh-CN: 系统信息
en-US: System Info
priority: 0
protocol: http
http:
host: ^_^host^_^
port: ^_^port^_^
ssl: ^_^ssl^_^
url: /ISAPI/System/deviceInfo
method: GET
timeout: ^_^timeout^_^
authorization:
type: Digest Auth
digestAuthUsername: ^_^username^_^
digestAuthPassword: ^_^password^_^
parseType: xmlPath
parseScript: //DeviceInfo
fields:
- field: deviceName
type: 1
i18n:
zh-CN: 设备名称
en-US: Device Name
- field: deviceID
type: 1
i18n:
zh-CN: 设备ID
en-US: Device ID
- field: firmwareVersion
type: 1
i18n:
zh-CN: 固件版本
en-US: Firmware Version
- field: model
type: 1
i18n:
zh-CN: 设备型号
en-US: Device Model
- field: macAddress
type: 1
i18n:
zh-CN: mac地址
en-US: Mac Address
- name: status
i18n:
zh-CN: 设备状态
en-US: Status
priority: 0
protocol: http
http:
host: ^_^host^_^
port: ^_^port^_^
ssl: ^_^ssl^_^
url: /ISAPI/System/status
method: GET
timeout: ^_^timeout^_^
authorization:
type: Digest Auth
digestAuthUsername: ^_^username^_^
digestAuthPassword: ^_^password^_^
parseType: xmlPath
parseScript: //DeviceStatus
fields:
- field: CPU_utilization
i18n:
zh-CN: CPU 利用率
en-US: CPU Utilization
type: 0
unit: '%'
- field: memory_usage
i18n:
zh-CN: 内存使用量
en-US: Memory Usage
type: 0
unit: MB
- field: memory_available
i18n:
zh-CN: 可用内存
en-US: Memory Available
type: 0
unit: MB
- field: cache_size
i18n:
zh-CN: 缓存大小
en-US: Cache Size
type: 0
unit: MB
- field: net_port_1_speed
i18n:
zh-CN: 网口1速度
en-US: Net Port 1 Speed
type: 0
unit: Mbps
- field: net_port_2_speed
i18n:
zh-CN: 网口2速度
en-US: Net Port 2 Speed
type: 0
unit: Mbps
- field: boot_time
i18n:
zh-CN: 启动时间
en-US: Boot Time
type: 1
- field: device_uptime
i18n:
zh-CN: 运行时长
en-US: Device Uptime
type: 1
- field: last_calibration_time
i18n:
zh-CN: 上次校时时间
en-US: Last Calibration Time
type: 1
- field: last_calibration_time_diff
i18n:
zh-CN: 上次校时时间差
en-US: Last Calibration Time Diff
type: 0
unit: s
- field: avg_upload_time
i18n:
zh-CN: 平均上传耗时
en-US: Avg Upload Time
type: 0
unit: ms
- field: max_upload_time
i18n:
zh-CN: 最大上传耗时
en-US: Max Upload Time
type: 0
unit: ms
- field: min_upload_time
i18n:
zh-CN: 最小上传耗时
en-US: Min Upload Time
type: 0
unit: ms
- field: last_calibration_mode
i18n:
zh-CN: 上次校时模式
en-US: Last Calibration Mode
type: 1
- field: last_calibration_address
i18n:
zh-CN: 上次校时地址
en-US: Last Calibration Address
type: 1
- field: response_time
i18n:
zh-CN: 响应时间
en-US: Response Time
type: 0
unit: ms
aliasFields:
- CPUList/CPU/cpuUtilization
- MemoryList/Memory/memoryUsage
- MemoryList/Memory/memoryAvailable
- MemoryList/Memory/cacheSize
- NetPortStatusList/NetPortStatus[id='1']/workSpeed
- NetPortStatusList/NetPortStatus[id='2']/workSpeed
- bootTime
- deviceUpTime
- lastCalibrationTime
- lastCalibrationTimeDiff
- uploadTimeConsumingList/avgTime
- uploadTimeConsumingList/maxTime
- uploadTimeConsumingList/minTime
- lastCalibrationTimeMode
- lastCalibrationTimeAddress
- responseTime
calculates:
- CPU_utilization=CPUList/CPU/cpuUtilization
- memory_usage=MemoryList/Memory/memoryUsage
- memory_available=MemoryList/Memory/memoryAvailable
- cache_size=MemoryList/Memory/cacheSize
- net_port_1_speed=NetPortStatusList/NetPortStatus[id='1']/workSpeed
- net_port_2_speed=NetPortStatusList/NetPortStatus[id='2']/workSpeed
- boot_time=bootTime
- device_uptime=deviceUpTime
- last_calibration_time=lastCalibrationTime
- last_calibration_time_diff=lastCalibrationTimeDiff
- avg_upload_time=uploadTimeConsumingList/avgTime
- max_upload_time=uploadTimeConsumingList/maxTime
- min_upload_time=uploadTimeConsumingList/minTime
- last_calibration_mode=lastCalibrationTimeMode
- last_calibration_address=lastCalibrationTimeAddress
- response_time=responseTime
units:
- memory_usage=KB->MB
- memory_available=KB->MB
- cache_size=KB->MB
+2 -2
View File
@@ -62,8 +62,8 @@ The HertzBeat install package will at `dist/hertzbeat-{version}.tar.gz`
2. Execute under the project root directory: `mvn clean install`
3. Cd to the `hertzbeat-collector` directory: `cd hertzbeat-collector`
3. Cd to the `collector` directory: `cd collector`
4. Execute under `hertzbeat-collector` directory: `mvn clean package -Pcluster`
4. Execute under `collector` directory: `mvn clean package -Pcluster`
The HertzBeat collector package will at `dist/hertzbeat-collector-{version}.tar.gz`
+1 -7
View File
@@ -215,7 +215,6 @@ The release package are here:
- `dist/apache-hertzbeat-{version}-incubating-bin.tar.gz`
- `dist/apache-hertzbeat-collector-{version}-incubating-bin.tar.gz`
- `dist/apache-hertzbeat-{version}-incubating-docker-compose.tar.gz`
#### 3.4 Package the source code
@@ -258,9 +257,6 @@ apache-hertzbeat-1.6.0-incubating-src.tar.gz.sha512
apache-hertzbeat-1.6.0-incubating-bin.tar.gz
apache-hertzbeat-1.6.0-incubating-bin.tar.gz.asc
apache-hertzbeat-1.6.0-incubating-bin.tar.gz.sha512
apache-hertzbeat-1.6.0-incubating-docker-compose.tar.gz
apache-hertzbeat-1.6.0-incubating-docker-compose.tar.gz.asc
apache-hertzbeat-1.6.0-incubating-docker-compose.tar.gz.sha512
apache-hertzbeat-collector-1.6.0-incubating-bin.tar.gz
apache-hertzbeat-collector-1.6.0-incubating-bin.tar.gz.asc
apache-hertzbeat-collector-1.6.0-incubating-bin.tar.gz.sha512
@@ -294,8 +290,6 @@ apache-hertzbeat-1.6.0-incubating-src.tar.gz
apache-hertzbeat-1.6.0-incubating-src.tar.gz: OK
apache-hertzbeat-1.6.0-incubating-bin.tar.gz
apache-hertzbeat-1.6.0-incubating-bin.tar.gz: OK
apache-hertzbeat-1.6.0-incubating-docker-compose.tar.gz
apache-hertzbeat-1.6.0-incubating-docker-compose.tar.gz: OK
apache-hertzbeat-collector-1.6.0-incubating-bin.tar.gz
apache-hertzbeat-collector-1.6.0-incubating-bin.tar.gz: OK
```
@@ -341,7 +335,7 @@ svn commit -m "release for HertzBeat 1.6.0"
- Check Apache SVN Commit Results
> Visit the address <https://dist.apache.org/repos/dist/dev/incubator/hertzbeat/> in the browser, check if existed the new material package
> Visit the address <https://dist.apache.org/repos/dist/dev/incubator/hertzbeat/1.6.0-RC1/> in the browser, check if existed the new material package
## 4. Enter the community voting stage
+3 -4
View File
@@ -20,10 +20,9 @@ sidebar_label: Download
Previous releases of HertzBeat may be affected by security issues, please use the latest one.
:::
| Version | Date | Download | Release Notes |
| ------- | ---------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| v1.7.0 | 2025.04.02 | [apache-hertzbeat-1.7.0-incubating-bin.tar.gz](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.7.0/apache-hertzbeat-1.7.0-incubating-bin.tar.gz) (HertzBeat) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.7.0/apache-hertzbeat-1.7.0-incubating-bin.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.7.0/apache-hertzbeat-1.7.0-incubating-bin.tar.gz.sha512) ) <br/> [apache-hertzbeat-collector-1.7.0-incubating-bin.tar.gz](https://downloads.apache.org/incubator/hertzbeat/1.7.0/apache-hertzbeat-collector-1.7.0-incubating-bin.tar.gz) (HertzBeat Collector) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.7.0/apache-hertzbeat-collector-1.7.0-incubating-bin.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.7.0/apache-hertzbeat-collector-1.7.0-incubating-bin.tar.gz.sha512) ) <br/> [apache-hertzbeat-1.7.0-incubating-src.tar.gz](https://downloads.apache.org/incubator/hertzbeat/1.7.0/apache-hertzbeat-1.7.0-incubating-src.tar.gz) (HertzBeat Source) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.7.0/apache-hertzbeat-1.7.0-incubating-src.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.7.0/apache-hertzbeat-1.7.0-incubating-src.tar.gz.sha512) ) <br/> [apache-hertzbeat-1.7.0-incubating-docker-compose.tar.gz](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.7.0/apache-hertzbeat-1.7.0-incubating-docker-compose.tar.gz) (docker-compose) ( [signature](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.7.0/apache-hertzbeat-1.7.0-incubating-docker-compose.tar.gz.asc) , [sha512](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.7.0/apache-hertzbeat-1.7.0-incubating-docker-compose.tar.gz.sha512) ) | [release note](https://github.com/apache/hertzbeat/releases/tag/v1.7.0) |
| v1.6.1 | 2024.10.29 | [apache-hertzbeat-1.6.1-incubating-bin.tar.gz](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-bin.tar.gz) (HertzBeat) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-bin.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-bin.tar.gz.sha512) ) <br/> [apache-hertzbeat-collector-1.6.1-incubating-bin.tar.gz](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-collector-1.6.1-incubating-bin.tar.gz) (HertzBeat Collector) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-collector-1.6.1-incubating-bin.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-collector-1.6.1-incubating-bin.tar.gz.sha512) ) <br/> [apache-hertzbeat-1.6.1-incubating-src.tar.gz](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-src.tar.gz) (HertzBeat Source) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-src.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-src.tar.gz.sha512) ) <br/> [apache-hertzbeat-1.6.1-incubating-docker-compose.tar.gz](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-docker-compose.tar.gz) (docker-compose) ( [signature](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-docker-compose.tar.gz.asc) , [sha512](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-docker-compose.tar.gz.sha512) ) | [release note](https://github.com/apache/hertzbeat/releases/tag/v1.6.1) |
| Version | Date | Download | Release Notes |
|---------|------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|
| v1.6.1 | 2024.10.29 | [apache-hertzbeat-1.6.1-incubating-bin.tar.gz](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-bin.tar.gz) (HertzBeat) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-bin.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-bin.tar.gz.sha512) ) <br/> [apache-hertzbeat-collector-1.6.1-incubating-bin.tar.gz](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-collector-1.6.1-incubating-bin.tar.gz) (HertzBeat Collector) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-collector-1.6.1-incubating-bin.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-collector-1.6.1-incubating-bin.tar.gz.sha512) ) <br/> [apache-hertzbeat-1.6.1-incubating-src.tar.gz](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-src.tar.gz) (HertzBeat Source) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-src.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-src.tar.gz.sha512) ) <br/> [apache-hertzbeat-1.6.1-incubating-docker-compose.tar.gz](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-docker-compose.tar.gz) (docker-compose) ( [signature](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-docker-compose.tar.gz.asc) , [sha512](https://dist.apache.org/repos/dist/release/incubator/hertzbeat/1.6.1/apache-hertzbeat-1.6.1-incubating-docker-compose.tar.gz.sha512) ) | [release note](https://github.com/apache/hertzbeat/releases/tag/v1.6.1)|
| v1.6.0 | 2024.06.10 | [apache-hertzbeat-1.6.0-incubating-bin.tar.gz](https://downloads.apache.org/incubator/hertzbeat/1.6.0/apache-hertzbeat-1.6.0-incubating-bin.tar.gz) (HertzBeat) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.6.0/apache-hertzbeat-1.6.0-incubating-bin.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.6.0/apache-hertzbeat-1.6.0-incubating-bin.tar.gz.sha512) ) <br/> [apache-hertzbeat-collector-1.6.0-incubating-bin.tar.gz](https://downloads.apache.org/incubator/hertzbeat/1.6.0/apache-hertzbeat-collector-1.6.0-incubating-bin.tar.gz) (HertzBeat Collector) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.6.0/apache-hertzbeat-collector-1.6.0-incubating-bin.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.6.0/apache-hertzbeat-collector-1.6.0-incubating-bin.tar.gz.sha512) ) <br/> [apache-hertzbeat-1.6.0-incubating-src.tar.gz](https://downloads.apache.org/incubator/hertzbeat/1.6.0/apache-hertzbeat-1.6.0-incubating-src.tar.gz) (HertzBeat Source) ( [signature](https://downloads.apache.org/incubator/hertzbeat/1.6.0/apache-hertzbeat-1.6.0-incubating-src.tar.gz.asc) , [sha512](https://downloads.apache.org/incubator/hertzbeat/1.6.0/apache-hertzbeat-1.6.0-incubating-src.tar.gz.sha512) ) | [release note](https://github.com/apache/hertzbeat/releases/tag/v1.6.0) |
## Release Docker Image
-57
View File
@@ -1,57 +0,0 @@
---
id: alarm_center
title: Alarm Center
sidebar_label: Alarm Center
keywords:
[open-source monitoring system, alarm center, alarm management, alarm display]
---
> The Alarm Center serves as a comprehensive visualization platform that displays all alarms after undergoing grouping, consolidation, suppression, and silencing processes. It encompasses both internally triggered threshold-based alarms and integrated third-party notifications.
## Alarm Sources
The HertzBeat Alarm Center manages notifications from two primary sources:
1. Internal Threshold-Triggered Alarms
- Generated when monitoring metrics exceed predefined thresholds
- Directly correlated with monitoring tasks and threshold rules configured within the system
- Manageable through adjustment of monitoring parameters and threshold configurations
2. Third-Party Integrated Alarms
- Received through API interfaces from external systems
- Compatible with various monitoring systems and alarm platforms
- Processed through identical workflow as internal alarms
## Alarm Processing Mechanism
Before appearing in the Alarm Center, all notifications undergo several processing stages:
1. Grouping
- Categorizes related alarms based on source, type, severity, and other attributes (labels)
- Facilitates efficient management of high-volume alarms
- Supports customizable grouping rules for diverse scenarios
2. Consolidation
- Mitigates notification fatigue from multiple similar alarms within short intervals
- Presents consolidated alarms in a streamlined format, eliminating redundancy
3. Suppression
- Manages alarm dependencies
- Suppresses secondary alarms when primary alarms are triggered
- Supports configurable suppression rules based on alarm dependencies
4. Silencing
- Temporarily mutes specific alarms during designated periods
- Ideal for system maintenance windows and known issue handling
- Enables time-based silence rule configuration
## Alarm Center Interface
![alarm_center](/img/docs/help/alarm-center-1.png)
The Alarm Center provides a comprehensive view of all system alarms:
1. Alarm Display
- Lists all alarms with crucial information including status, source, labels, and timestamps
- Offers detailed view functionality for comprehensive alarm information and context
2. Search Functionality
- Enables rapid alarm identification
- Supports multiple search criteria (labels, annotations, alarm status)
3. Alarm Management
- Alarm Deletion: Removes alarms no longer requiring attention
-24
View File
@@ -1,24 +0,0 @@
---
id: alarm_group
title: Alarm Grouping
sidebar_label: Alarm Grouping
keywords: [Open source monitoring system, alarm reduce, alarm grouping]
---
> Group convergence supports grouping and convergence of alarms for specified packet labels, deduplication and convergence of the same repeated alarms for the time period. When the threshold rule triggers the alarm or external alarm reporting, it will enter the packet convergence to alarm grouping to deduplicate the alarm to avoid a large number of alarm messages causing alarm storms.
## Grouping Policy Parameter Configuration
- Strategy Name: The name that uniquely identifies the grouping policy
- Group Labels: Alarm grouping tag, support up to 10 tags
> Tag source: monitoring, threshold rules, tags carried by external alarms
- Wait Time: Waiting time after a new alarm is generated. The same alarms received during this time will be grouped, with a default of 30 seconds.
> When a new (unable to join an existing group) alarm is generated, the group convergence will wait according to the `wait time`, during which time, the same alarm or the alarm that meets the grouping conditions will be grouped. The alarm after the grouping is sent to the alarm suppression module for subsequent processing until the time interval between the current time and the first alarm generation in the packet exceeds the `wait time`.
- Interval time: The minimum time interval for sending group alarm notifications to avoid excessive alarm notifications, default 5 minutes
- Repeat interval: The minimum notification interval for repeated alarms. For continuously triggered alarms, avoid repeated notifications, default 4 hours
**Note**: Only grouped alarms can be suppressed using suppression rules.

Some files were not shown because too many files have changed in this diff Show More