包括filebeat镜像和应用服务镜像
根据自己的服务配置加入以下内容
---
apiVersion: v1
kind: Service
metadata:
name: test-app
labels:
app: test-app
spec:
selector:
app: test-app
ports:
- protocol: TCP
port: 8901
targetPort: 8901
name: test-port
---
apiVersion: v1
kind: ConfigMap
metadata:
name: test-filebeat-config
labels:
k8s-app: filebeat
data:
filebeat.yml: |-
filebeat.prospectors:
- type: log
paths:
- /logdata/*.log
tail_files: true
fields:
pod_name: '${pod_name}'
POD_IP: '${POD_IP}'
setup.template.name: "app-logs"
setup.template.pattern: "app-logs-*"
output.kafka:
hosts: ["134.64.15.155:9092"]
topic: kafka_log_sidecar
partition.round_robin:
reachable_only: true
# deployment, 也可通过daemonset方式
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-app
spec:
replicas: 1
minReadySeconds: 15 #滚动升级15s后标志pod准备就绪
strategy:
rollingUpdate: #replicas为2, 升级过程中pod个数在1-3个之间
maxSurge: 1 #滚动升级时会先启动1个pod
maxUnavailable: 1 #滚动升级时允许pod处于Unavailable的最大个数
selector:
matchLabels:
app: test-app
template:
metadata:
labels:
app: test-app
spec:
terminationGracePeriodSeconds: 30 #30秒内优雅关闭程序
containers:
- image: 134.64.15.155:5000/docker.elastic.co/beats/filebeat:6.2.4 #提前下载下来到私有镜像库的镜像(官方的可能会被墙)
name: filebeat
args: [
"-c", "/opt/filebeat/filebeat.yml",
"-e",
]
env:
- name: POD_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: pod_name
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
securityContext:
runAsUser: 0
resources:
limits:
memory: 200Mi
requests:
cpu: 200m
memory: 200Mi
volumeMounts:
- name: config #将configmap的内容放到容器本地目录
mountPath: /opt/filebeat/
- name: data
mountPath: /usr/share/filebeat/data
- name: logdata #同一个pod内的两个应用共享目录logdata, 一个写一个读
mountPath: /logdata
- name: test-app
image: 134.64.15.155:5000/kuberneteservice-dev:v1 #提供具体服务的app镜像
command: ["nohup","sh", "KuberneteService", "&"]
ports:
- containerPort: 8901
volumeMounts:
- name: logdata #指定挂在目录到logdata
mountPath: /log
volumes:
- name: data
emptyDir: {}
- name: logdata #定义logdata为EmptyDir类型挂载目录
emptyDir: {}
- name: config
configMap:
name: test-filebeat-config #使用前面定义的configmap
items:
- key: filebeat.yml
path: filebeat.yml
kubectl apply -f sidecar.yml
kubectl get all | grep test-app
如果运行失败可以查看运行情况，删除应用重新部署
查看失败详情，会在最后一行显示问题详情：
kubectl describe pods pod名
删除部署应用：
kubectl delete service test-app
kubectl delete configmap test-filebeat-config
kubectl delete deployment.apps test-app
kubectl exec -it pod名 -c test-app /bin/bash
kubectl exec -it pod名 -c filebeat /bin/bash
查看filebeat日志：
kubectl logs -f pod名 -c filebeat
查看应用日志：
kubectl logs -f pod名 -c test-app
./kafka-console-consumer.sh --bootstrap-server 134.64.15.155:9092 --topic kafka_log_sidecar
数据格式：
{
"@timestamp": "2022-12-07T03:04:07.290Z",
"@metadata": {
"beat": "filebeat",
"type": "doc",
"version": "6.2.4",
"topic": "kafka_log_sidecar"
},
"source": "/logdata/out.log",
"offset": 9464,
"message": "2022-12-07 03:04:00.001 ERROR [schedulerFactoryBean_Worker-6] c.c.u.c.d.c.RefreshWorker {\"Message\":\"engine is null,please check engine is avaliable.\",\"User\":\"root\",\"Name\":\"cmdb\"}",
"prospector": {
"type": "log"
},
"fields": {
"POD_IP": "10.240.4.59",
"pod_name": "test-app-7d646dd5b4-zr9h4"
},
"beat": {
"name": "test-app-7d646dd5b4-zr9h4",
"hostname": "test-app-7d646dd5b4-zr9h4",
"version": "6.2.4"
}
}
报错内容：Failed to connect to broker [:9092 dial tcp: lookup on server misbehaving
解决：
修改server.properties配置文件，在配置文件里添加这两项：
listeners=PLAINTEXT://xx.xx.xx.xx:9092
advertised.listeners=PLAINTEXT://xx.xx.xx.xx:9092
报错内容：Kubernetes之network: failed to set bridge addr: “cni0“ already has an IP address different from xxx问题
解决：
去pod所在主机执行以下命令：
sudo ifconfig cni0 down
sudo ip link delete cni0
从上述表格中可以看出：
原生方式相对功能太弱，一般不建议在生产系统中使用，否则问题调查、数据统计等工作很难完成；
DaemonSet方式在每个节点只允许一个日志agent，相对资源占用要小很多，但扩展性、租户隔离性受限，比较适用于功能单一或业务不是很多的集群；
Sidecar方式为每个POD单独部署日志agent，相对资源占用较多，但灵活性以及多租户隔离性较强，建议大型的K8S集群或作为PAAS平台为多个业务方服务的集群使用该方式。