[EKS] How to deploy an application with Kubernetes

How to use K8S with EKS

EKS란 AWS클라우드 환경에서 k8s를 Managed 서비스로 제공하는 서비스이다.
아래 튜토리얼을 통해 기본적인 클러스터 생성 및 어플리케이션 배포까지의 Step을 진행하였다.

Installing k8s with EKS Tools

# installing aws-cli
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
aws --version

# installing eksctl
curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin
eksctl version

# installing kubectl
curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.19.6/2021-01-05/bin/linux/amd64/kubectl
chmod +x ./kubectl
mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
kubectl version --short --client

Creating a EKS Cluster

EKS Cluster 생성을 위한 node(computing)의 타입, 볼륨, 노드 수 등을 지정한다.

# creating eks cluster
eksctl create cluster \
    --name k8s-demo \
    --region ap-northeast-2 \
    --with-oidc \
    --ssh-access \
    --ssh-public-key rocksea-bastion \
    --nodes 3 \
    --node-type t3.medium \
    --node-volume-size=20 \
    --managed

# get .kube configuration from eks
aws eks --region ap-northeast-2 update-kubeconfig --name k8s-demo

# get cluster list
kubectl get all

Setting ConfigMap

• configmap.yaml

ConfigMap을 통해 시스템 환경변수를 등록하여 관리한다.

apiVersion: v1
kind: ConfigMap
metadata:
  name: larambla-config
data:
  dbHost: 'larambla.ap-northeast-2.rds.amazonaws.com'
  dbPort: '3306'
  dbName: 'larambla'
  dbUsername: 'larambla'
  dbPassword: 'larambla'
  dbDdlauto: 'update'

• configmap

kubectl apply -f configmap.yaml

Creating Secrets

Password와 같은 기민한 정보는 Screts를 통해 base64 encoding을 하여 보관한다.

kubectl create secret generic spring-security \
--from-literal=spring.user.name=demo \
--from-literal=spring.user.password=password

Deploying a Application

• deployment.yaml

배포할 Pod의 Replicas(각 node에 복제 할 갯 수), Container Image, Inner Service Port 등을 지정한다.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: larambla-deployment
  labels:
    app: java-microservice
spec:
  replicas: 3
  selector:
    matchLabels:
      app: larambla-member-api
  template:
    metadata:
      labels:
        app: larambla-member-api
    spec:
      containers:
      - name: larambla
        image: xxxxxxxxxx.dkr.ecr.ap-northeast-2.amazonaws.com/larambla:latest
        ports:
        - containerPort: 5000
    env:
      - name: DB_HOST
        valueFrom:
          configMapKeyRef:
            name: larambla-config
            key: dbHost
      - name: DB_PORT
        valueFrom:
          configMapKeyRef:
            name: larambla-config
            key: dbPort
      - name: DB_NAME
        valueFrom:
          configMapKeyRef:
            name: larambla-config
            key: dbName
      - name: DB_USERNAME
        valueFrom:
          configMapKeyRef:
            name: larambla-config
            key: dbUsername
      - name: DB_PASSWORD
        valueFrom:
          configMapKeyRef:
            name: larambla-config
            key: dbPassword
      - name: DB_DDL_AUTO
        valueFrom:
          configMapKeyRef:
            name: larambla-config
            key: dbDdlauto

• service.yaml

배포 된 Pod에 로드밸런싱을 위한 서비스 Cluster 단위를 구성한다. 해당 서비스 생성을 통해 selector로 지정한 application이 실행중인 pod에 트래픽을 로드밸런싱을 한다.

apiVersion: v1
kind: Service
metadata:
  name: larambla-service
spec:
  #Creating a service of type load balancer. Load balancer gets created but takes time to reflect
  type: LoadBalancer
  selector:
    app: larambla-member-api
  ports:
  - protocol: TCP
    port: 80
    targetPort: 5000

• EKS Command

# deploying applications 
kubectl apply -f deployment.yaml

# deploying services 
kubectl apply -f service.yaml

# setting auto-scale 
kubectl autoscale deployment larambla-deployment --cpu-percent=50 --min=3 --max=5

Installing a Ingress Controller

• Installing ingress controller

각 서비스에 라우팅 기능을 이용하기 위해 ingress controller를 설치한다.

helm install ingress-nginx-2 ingress-nginx/ingress-nginx
--namespace ingress-nginx-2
--set controller.ingressClassResource.name=nginx-2
--set controller.ingressClassResource.controllerValue="k8s.io/ingress-nginx-2"
--set controller.ingressClassResource.enabled=true
--set controller.ingressClassByName=true

• ingress.yaml

ingress controller를 이용하여 각 서비스 라우팅 전략을 기술하고 적용한다. nginx의 reverse proxy와 같이 각 호스트의 path별 라우팅 대상 서비스를 지정 할 수 있다.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-larambla
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: larambla.co.kr
    http:
      paths:
      - path: /member
        pathType: Prefix
        backend:
          service:
            name: larambla-service
            port:
              number: 80
  ingressClassName: nginx

• Creating Ingress

kubectl apply -f ingress.yaml

Installing a Dashboard

• account.yaml

Dashboard에 access 가능한 service account를 생성한다.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

대쉬보드를 통해 워크로드, 노드, 서비스, 파드 등의 상태를 모니터링 및 관리할 수 있다.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.4.0/aio/deploy/recommended.yaml
kubectl apply -f user.yaml
kubectl -n kubernetes-dashboard get secret $(kubectl -n kubernetes-dashboard get sa/admin-user -o jsonpath="{.secrets[0].name}") -o go-template="{{.data.token | base64decode}}"

Proxy를 통해 대시보드 내부에 Access 할 수 있다.

kubectl proxy

http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/workloads?namespace=default

Connecting Aurora Mysql Client

Pod를 통해 Private Network의 DB에 접근도 가능하다.

kubectl run -it --rm --image=mysql:5.7 --restart=Never mysql-client -- mysql -ularambla -plarambla -hdb.ap-northeast-2.rds.amazonaws.com larambla

Adding RBAC with System Master

EKS Cluster생성 계정외에 추가로 Admin Role에 EKS접근 권한을 부여하였다.

kubectl edit -n kube-system configmap/aws-auth

apiVersion: v1
data:
  mapRoles: |
    - groups:
      - system:bootstrappers
      - system:nodes
      rolearn: arn:aws:iam::xxxxxxxxxxxx:role/eksctl-larambla-cluster-nodegroup-NodeInstanceRole-7NCDI3TA146Z
      username: system:node:{{EC2PrivateDNSName}}
    **- groups:
      - system:masters
      rolearn: arn:aws:iam::xxxxxxxxxxxx:role/Admin
      username: rocksea**

References

• AWS WORKSHOP
  • https://awskrug.github.io/eks-workshop/introduction/
  • https://www.eksworkshop.com/010_introduction/
• Dashboard
  • https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md
• Installation Ingress Controller
  • https://kubernetes.github.io/ingress-nginx/
  • https://kubernetes.github.io/ingress-nginx/user-guide/basic-usage/
  • https://kubernetes.github.io/ingress-nginx/examples/rewrite/
  • https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.5/deploy/static/provider/cloud/deploy.yaml
• Deploying to EKS
  • JAVA - https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/deploy-java-microservices-on-amazon-eks-by-using-amazon-ecr-and-eksctl.html
  • PHP - https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/
• MySQL
  • https://kubernetes.io/docs/tasks/run-application/run-single-instance-stateful-application/
• ArgoCD
  • https://argo-cd.readthedocs.io/en/stable/cli_installation/