Skip to content

Kubernetes 生态链路整合

第一章：kustomize

1.1 概述

  • kustomize 是 Kubernetes 本地的配置工具，相当于轻量级的 Helm 。
  • kustomize 可以快速部署不同环境（开发、测试、预生产、生产等）的应用，只需要通过 kubectl apply -k 命令即可。

1.2 应用实例

1681265469844-dd5143de-3060-49c8-b64d-7a4acc779baf.png

  • base/configMap.yaml
kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configmap
data:
  index.html: "<!DOCTYPE html>\r\n<html lang=\"en\">\r\n<head>\r\n    <meta charset=\"UTF-8\">\r\n    <meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\">\r\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">\r\n    <title>Document</title>\r\n</head>\r\n<body>\r\n    你好，Nginx!!!\r\n</body>\r\n</html>"
  • base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment
spec:
  selector:
    matchLabels:
      app: deployment
  replicas: 1
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: deployment
    spec:
      containers:
      - name: deployment
        image: nginx:latest
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 100m
            memory: 100Mi
        livenessProbe:
          tcpSocket:
            port: 80
          initialDelaySeconds: 5
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 3
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 5
          timeoutSeconds: 2
          successThreshold: 1
          failureThreshold: 3
          periodSeconds: 10
        ports:
        - containerPort: 80
          name: deployment
        volumeMounts:
        - name: localtime
          mountPath: /etc/localtime
      volumes:
      - name: localtime
        hostPath:
          path: /usr/share/zoneinfo/Asia/Taipei
      restartPolicy: Always
  • base/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  selector:
    app: deployment
  type: ClusterIP
  ports:
  - name: nginx
    protocol: TCP
    port: 80
    targetPort: 80
  • base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
commonLabels:
  app: nginx # 构建出来的每个资源上都有 app=nginx 标签
resources: # 需要加载的资源
- configMap.yaml
- deployment.yaml
- service.yaml
  • overlays/development/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment
spec: # 只需要定义可变部分
  replicas: 3
  • overlays/development/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namePrefix: development- # 开发环境所有资源加前缀
commonLabels:
  variant: development
commonAnnotations:
  note: 大家好，这是开发环境！！！
bases: # 所有资源的基本位置
- ../../base
patchesStrategicMerge: # 增量合并
- deployment.yaml
  • overlays/production/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment
spec: # 只需要定义可变部分
  replicas: 7
  • overlays/production/kustomization.yaml
# $BASE/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namePrefix: production- # 生产环境所有资源加前缀
commonLabels:
  variant: production
commonAnnotations:
  note: 大家好，这是生产环境！！！
bases:
- ../../base
patchesStrategicMerge:
- deployment.yaml
  • 安装基础配置的资源：
kubectl apply -k base

注意：

  • ① 可以在安装的时候指定 namespace ，即 kubectl apply -k base -n dev。
  • ② 删除可以使用 delete ，即 kubectl delete -k base 。

1681265536569-0bb1b03d-0772-4de6-bd7c-37cf3c51674b.gif

  • 安装开发环境和生产环境的资源：
kubectl apply -k overlays/development

kubectl apply -k overlays/production

1681265542980-f51cdae6-1fd9-42e6-abfb-9e6058417a7c.gif

第二章：ECK（Elastic Cloud on Kubernetes，Helm）

2.1 安装 Operator

  • 创建目录并进入目录：
mkdir eck && cd eck

1681265551447-a3f87780-480b-46b5-9810-4cb8cb552a82.gif

  • 生成证书：
openssl req -x509 -nodes -days 3650 -newkey rsa:4096 -keyout tls.key -out tls.crt -subj "/CN=*.xudaxian.com/O=*.xudaxian.com" \
  -extensions san \
  -config <(echo '[req]'; echo 'distinguished_name=req';
            echo '[san]'; echo 'subjectAltName=DNS:*.xudaxian.com,DNS:www.xudaxian.com,DNS:harbor.xudaxian.com,DNS:alertmanager.xudaxian.com,DNS:grafana.xudaxian.com,DNS:prometheus.xudaxian.com,DNS:jenkins.xudaxian.com,DNS:elastic.xudaxian.com,DNS:kibana.xudaxian.com,DNS:filebeats.xudaxian.com,DNS:thanos-gateway.xudaxian.com,DNS:gitlab.xudaxian.com')

kubectl create ns eck

kubectl create secret tls xudaxian.com --cert=tls.crt --key=tls.key  -n eck

注意：实际生产环境中需要自己购买域名。

1681265557145-09ff59e8-5503-401f-80ce-bade2de1ff12.gif

  • 增加仓库：
helm repo add elastic https://helm.elastic.co

1681265665139-f6cd023e-2a2c-4568-97be-5e0c109e3607.gif

  • 更新仓库索引：
helm repo update

1681265669980-c056f038-eef4-4665-b7db-7dd2c7cdb72a.gif

  • 集群范围（全局）安装：
helm install elastic-operator elastic/eck-operator -n elastic-system --create-namespace

1681265677219-04f134a7-5faa-4282-90cc-d57f63798222.gif

  • 受限安装（可选）：限制仅管理一组预定义的命名空间
helm install elastic-operator-crds elastic/eck-operator-crds

helm install elastic-operator elastic/eck-operator -n elastic-system --create-namespace \
  --set=installCRDs=false \
  --set=managedNamespaces='{namespace-a, namespace-b}' \
  --set=createClusterScopedResources=false \
  --set=webhook.enabled=false \
  --set=config.validateStorageClass=false

2.2 安装 ElasticSearch

  • 创建目录并进入目录：
mkdir eck-elasticsearch && cd eck-elasticsearch

1681265719075-3fe7cfbb-bc22-4988-bddf-4a9f48d95173.gif

  • 获取 eck-elasticsearch 的 Chart 包：
helm pull elastic/eck-elasticsearch --version 0.3.0

1681265723080-5f2c1e6f-9418-4f2a-b400-dc091fbf0018.gif

  • 获取 eck-elasticsearch 的默认值：
helm show values eck-elasticsearch-0.3.0.tgz > override.yaml

1681265726909-7239e22a-5281-42d7-bfab-8cf6d0d9763c.gif

  • 修改 eck-elasticsearch 的 override.yaml 的内容：
vim override.yaml

version: 8.7.0
annotations:
  eck.k8s.elastic.co/license: basic
nodeSets:
- name: masters
  count: 3
  config:
    node.roles: ["master"] 
    xpack.ml.enabled: true 
    node.store.allow_mmap: false
  volumeClaimTemplates:
  - metadata:
      name: elasticsearch-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 5Gi
      storageClassName: "ceph-block"
- name: data
  count: 5
  config:
    node.roles: ["data", "ingest", "ml", "transform"]
    node.store.allow_mmap: false
  volumeClaimTemplates:
  - metadata:
      name: elasticsearch-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 5Gi
      storageClassName: "ceph-block"

1681265732165-663dfa29-065a-43fe-a7ea-0310cc360fe2.gif

  • 安装：
helm install eck-elasticsearch eck-elasticsearch-0.3.0.tgz -f override.yaml \
        -n eck --create-namespace

1681265736485-6badc9f7-974a-4a6d-b9f4-588c4dfd611f.gif

  • 获取密码：
kubectl get secret -n eck

kubectl get secret eck-elasticsearch-es-elastic-user -o=jsonpath='{.data.elastic}' -n eck | base64 --decode; echo

注意：本人的密码是 vnPP9R8Amy8e78Q7Uo552p3K ，每个人的不一样！！！

1681265818903-b95e44cc-a32f-424f-969c-04dd579babe0.gif

  • 集群组件内访问：
# 账号是 elastic，密码上面步骤已经获取
curl -u "elastic:vnPP9R8Amy8e78Q7Uo552p3K" -k "https://eck-elasticsearch-es-http:9200"

curl -u "elastic:vnPP9R8Amy8e78Q7Uo552p3K" -k "https://10.96.120.162:9200"

1681265824978-38dea184-463c-44eb-bd16-8117aa001b63.gif

  • 部署 ingress 访问：
vim elastic-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: elastic-ingress
  namespace: eck
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/server-snippet: |
      proxy_ssl_verify off;
spec:
  tls:
  - hosts:
      - elastic.xudaxian.com
    secretName: xudaxian.com
  rules:
  - host: elastic.xudaxian.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: eck-elasticsearch-es-http
            port:
              number: 9200

kubectl apply -f elastic-ingress.yaml

1681265834572-95a9c51e-423d-4a8d-9e33-e51c51ebc0a3.gif

  • 配置域名解析：

1681265838570-8a12c703-306c-4b60-a02c-5ec7a2f243b2.png

  • 浏览器访问：

1681265848392-98e8922c-6140-4729-87ab-f50fcebf195e.gif

2.3 安装 Kibana

  • 创建目录并进入目录：
cd .. && mkdir -pv eck-kibana && cd eck-kibana

1681265853338-07e6ba39-2805-49e9-af91-6338f168ebbc.gif

  • 获取 eck-kibana 的 Chart 包：
helm pull elastic/eck-kibana --version 0.3.0

1681265857945-45459927-6ce7-44d5-bfe4-13565efd814a.gif

  • 获取 eck-kibana 默认值：
helm show values eck-kibana-0.3.0.tgz > override.yaml

1681265863105-48a08c44-d7b2-44b9-865f-0bdb9cce9a33.gif

  • 修改 eck-kibana 默认值：
vim override.yaml

version: 8.7.0
annotations:
  eck.k8s.elastic.co/license: basic
spec:
  count: 1
  elasticsearchRef:
    name: eck-elasticsearch
    namespace: eck

1681265868449-41547c5b-3b9f-437d-805c-99158ccb2bfd.gif

  • 安装：
helm install eck-kibana eck-kibana-0.3.0.tgz -f override.yaml \
        -n eck --create-namespace

1681265873213-1911b7a4-7235-42c8-91ef-bc7c77a871fa.gif

  • 获取密码：
kubectl get secret eck-elasticsearch-es-elastic-user -o=jsonpath='{.data.elastic}' -n eck | base64 --decode; echo

注意：账号是 elastic ，密码是 vnPP9R8Amy8e78Q7Uo552p3K，每个人的不一样！！！

1681265877959-08781ff3-d5ac-4861-b53e-72a35b5edba1.gif

  • 部署 ingress 访问：
vim kibana-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kibana-ingress
  namespace: eck
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/server-snippet: |
      proxy_ssl_verify off;
spec:
  tls:
  - hosts:
      - kibana.xudaxian.com
    secretName: xudaxian.com
  rules:
  - host: kibana.xudaxian.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: eck-kibana-kb-http
            port:
              number: 5601

kubectl apply -f kibana-ingress.yaml

1681265883831-91aeaee3-d15a-4b2e-aa94-96272b36f090.gif

  • 配置域名解析：

1681265888560-f493a0ad-076c-4d1c-b353-85b1f9d00899.png

  • 浏览器访问：

1681265893169-b89ca49f-a042-49c4-97a5-ecc0c8acc584.gif

2.4 安装 Filebeat

  • 创建目录并进入目录：
cd .. && mkdir -pv eck-beats && cd eck-beats

1681265898003-c35194c6-be1c-46c6-a6eb-428cbb4222d1.gif

  • 获取 eck-beats 的 Chart 包：
helm pull elastic/eck-beats --version 0.2.0

1681265902750-2878a97b-756b-4435-bb08-ff1ce972bd23.gif

  • 获取 eck-beats 默认值：
helm show values eck-beats-0.2.0.tgz > override.yaml

1681265907227-c090e16f-916c-43d8-b03e-7c0470b4c840.gif

  • 修改 eck-beats 默认值：
vim override.yaml

version: 8.7.0
annotations:
  eck.k8s.elastic.co/license: basic
spec:
  type: "filebeat" # filebeat,metricbeat,heartbeat,auditbeat,packetbeat,journalbeat
  kibanaRef:
    name: eck-kibana
    namespace: eck
  elasticsearchRef:
    name: eck-elasticsearch
    namespace: eck
  daemonSet:
    podTemplate:
      spec:
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        securityContext:
          runAsUser: 0
        containers:
        - name: filebeat
          volumeMounts:
          - name: varlogcontainers
            mountPath: /var/log/containers
          - name: varlogpods
            mountPath: /var/log/pods
          - name: varlibdockercontainers
            mountPath: /var/lib/docker/containers
        volumes:
        - name: varlogcontainers
          hostPath:
            path: /var/log/containers
        - name: varlogpods
          hostPath:
            path: /var/log/pods
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
  config:
    filebeat.inputs:
    - type: container
      paths:
      - /var/log/containers/*.log

1681265912564-6e31313d-e36b-4505-b7bd-cb433fa1cd78.gif

  • 安装：
helm install eck-filebeat eck-beats-0.2.0.tgz -f override.yaml \
        -n eck --create-namespace

1681265917338-61032608-5057-4e42-a84a-dd31a5da7dc6.gif

更新: 2023-04-12 02:18:43
原文: https://www.yuque.com/fairy-era/yg511q/ou2c6ady5amhoqsm

Comments