K8S-Pod挂载数据卷权限问题导致程序无法写入

Driverli / 2023-07-19 / 原文

场景：需要将同一个服务的多个Pod副本的日志持久化到本地，为方便根据Pod的名称快速查看日志，在Pod挂载数据卷时，可以通过subPathExpr和env变量配合使用为每个Pod副本自动创建以Pod名称命名的文件夹
故障：Pod挂载数据卷时，挂载点如果不存在会自动创建，但自动创建的挂载点属主属组为root，业务进程使用appuser启动，导致程序无法进行写操作
解决：利用初始化容器，先自动创建挂载点，此时属主属组为root，然后使用command新增appuser用户，并对挂载点进行授权，业务容器启动时挂载点已经存在了，不会再自动创建，所以挂载点属主属组不会发生变化，以保证程序可以进行写操作

#1. 初始化容器部分
      initContainers:
      - name: volume-init
        image: alpine:3.18
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        volumeMounts:
        - name: tomcat-logs
          mountPath: /app/apache-tomcat-9.0.0.M15/logs
          subPathExpr: $(POD_NAME)/tomcat_logs
        - name: app-logs
          mountPath: /app/logs
          subPathExpr: $(POD_NAME)/app_logs
        - name: backup
          mountPath: /app/backup
          subPathExpr: $(POD_NAME)/backup
        command: ["sh", "-c", "adduser -D appuser; chown -R appuser.appuser /app; ls -l /app"]

#2. 完整的yaml文件
[root@master-176 ~]# cat ccas-tomcat.yaml 
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: ccas-tomcat-deploy
spec:
  replicas: 3
  selector:
    matchLabels:
      app: ccas-tomcat
  template:
    metadata:
      name: ccas-tomcat
      labels:
        app: ccas-tomcat
    spec:
      volumes:
      - name: webapps
        nfs:
          path: /app/data/kubernetes/ccas/tomcat/webapps
          server: 172.16.x.x
      - name: tomcat-logs
        hostPath:
          path: /app/data/ccas
          type: DirectoryOrCreate
      - name: app-logs
        hostPath:
          path: /app/data/ccas
          type: DirectoryOrCreate
      - name: backup
        hostPath:
          path: /app/data/ccas
          type: DirectoryOrCreate
      initContainers:
      - name: volume-init
        image: alpine:3.18
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        volumeMounts:
        - name: tomcat-logs
          mountPath: /app/apache-tomcat-9.0.0.M15/logs
          subPathExpr: $(POD_NAME)/tomcat_logs
        - name: app-logs
          mountPath: /app/logs
          subPathExpr: $(POD_NAME)/app_logs
        - name: backup
          mountPath: /app/backup
          subPathExpr: $(POD_NAME)/backup
        command: ["sh", "-c", "adduser -D appuser; chown -R appuser.appuser /app; ls -l /app"]
      containers:
      - name: tomcat
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        image: 172.16.x.x:18082/driverli/tomcat:9.0.0
        volumeMounts:
        - name: webapps
          mountPath: /app/apache-tomcat-9.0.0.M15/webapps
        - name: tomcat-logs
          mountPath: /app/apache-tomcat-9.0.0.M15/logs
          subPathExpr: $(POD_NAME)/tomcat_logs
        - name: app-logs
          mountPath: /app/logs
          subPathExpr: $(POD_NAME)/app_logs
        - name: backup
          mountPath: /app/backup
          subPathExpr: $(POD_NAME)/backup

---

kind: Service
apiVersion: v1
metadata:
  name: ccas-tomcat-svc
spec:
  type: NodePort
  selector:
    app: ccas-tomcat
  ports:
  - port: 8082
    protocol: TCP
    targetPort: 8080
    nodePort: 30086

#3. 创建资源
[root@master-176 ~]# kubectl apply -f ccas-tomcat.yaml

#4. 查看初始化容器日志
[root@master-176 ~]# kubectl logs ccas-tomcat-deploy-5c8dcd6755-5zg6j -c volume-init 
total 12
drwxr-xr-x    3 appuser  appuser       4096 Jul 19 06:25 apache-tomcat-9.0.0.M15
drwxr-xr-x    2 appuser  appuser       4096 Jul 19 06:25 backup
drwxr-xr-x    2 appuser  appuser       4096 Jul 19 06:25 logs
# 可以看到此时属组属组为appuser

#5. 检查程序日志是否可以写入
[root@worker-62 ~]# ll /app/data/ccas/ccas-tomcat-deploy-5c8dcd6755-qssf7/app_logs/ccas/
总用量 236
-rw-r-----. 1 appuser appuser  13634  7月 19 14:40 error.log
-rw-r-----. 1 appuser appuser 202608  7月 19 14:40 info.log
-rw-r-----. 1 appuser appuser  13634  7月 19 14:40 warn.log
[root@worker-62 ~]# ll /app/data/ccas/ccas-tomcat-deploy-5c8dcd6755-qssf7/tomcat_logs/
总用量 556
-rw-r-----. 1 appuser appuser  21705  7月 19 14:25 catalina.2023-07-19.log
-rw-r-----. 1 appuser appuser 489851  7月 19 14:40 catalina.out
-rw-r-----. 1 appuser appuser      0  7月 19 14:25 host-manager.2023-07-19.log
-rw-r-----. 1 appuser appuser    472  7月 19 14:28 localhost.2023-07-19.log
-rw-r-----. 1 appuser appuser  38451  7月 19 14:40 localhost_access_log.2023-07-19.txt
-rw-r-----. 1 appuser appuser      0  7月 19 14:25 manager.2023-07-19.log
# 可以看到已经有日志写入啦～