$ ls persistentvolumes/nfs/*.yaml | xargs -n 1 kubectl create -f
persistentvolume/pv001 created
persistentvolume/pv002 created
persistentvolume/pv003 created
persistentvolume/pv004 created
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv001 512Mi RWX Recycle Available 22s
pv002 128Mi RWX Recycle Available 22s
pv003 128Mi RWX Recycle Available 22s
pv004 128Mi RWX Recycle Available 22s
Persistent Volume Claim の用意
PV ができたら、PVC を作ります。
$ ls persistentvolumeclaims/*.yaml | xargs -n 1 kubectl create -f
persistentvolumeclaim/consul-config created
persistentvolumeclaim/consul-data created
persistentvolumeclaim/db-data created
persistentvolumeclaim/log-data created
$ kubectl get pv,pvc
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pv001 512Mi RWX Recycle Bound default/db-data 11m
persistentvolume/pv002 128Mi RWX Recycle Bound default/consul-data 11m
persistentvolume/pv003 128Mi RWX Recycle Bound default/log-data 11m
persistentvolume/pv004 128Mi RWX Recycle Bound default/consul-config 11m
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/consul-config Bound pv004 128Mi RWX 8s
persistentvolumeclaim/consul-data Bound pv002 128Mi RWX 8s
persistentvolumeclaim/db-data Bound pv001 512Mi RWX 8s
persistentvolumeclaim/log-data Bound pv003 128Mi RWX 7s
$ kompose convert -f docker-compose.yml
INFO Network edgex-network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination
...
INFO Kubernetes file "app-service-rules-service.yaml" created
...
INFO Kubernetes file "edgex-network-networkpolicy.yaml" created
名前解決したいホスト名が Service の名前と一致しているため、Kubernetes の ClusterIP に解決される
Pod 内のプロセスからは ClusterIP は利用できない（Pod の IP アドレスしか認識されない）ため、待ち受けできずに死ぬ
Consul 上のホスト名を書き換えるのも、Pod と Service でホスト名を別にするのも、副作用がわからないのでできれば避けたいところです。そもそも Kubernetes の仕様上、コンテナ内の /etc/hosts には Pod の IP アドレスと Pod のホスト名が正しく書いてあるわけですから、これを参照してくれさえすればよいだけの話です。
Cluster API is still in a prototype stage while we get feedback on the API types themselves. All of the code here is to experiment with the API and demo its abilities, in order to drive more technical feedback to the API design. Because of this, all of the codebase is rapidly changing.
$ cat envvars.txt
# vCenter config/credentials
export VSPHERE_SERVER='192.168.0.201' # (required) The vCenter server IP or FQDN
export VSPHERE_USERNAME='administrator@vsphere.local' # (required) The username used to access the remote vSphere endpoint
export VSPHERE_PASSWORD='my-secure-password' # (required) The password used to access the remote vSphere endpoint
# vSphere deployment configs
export VSPHERE_DATACENTER='kuro-dc01' # (required) The vSphere datacenter to deploy the management cluster on
export VSPHERE_DATASTORE='nfs-ds01' # (required) The vSphere datastore to deploy the management cluster on
export VSPHERE_NETWORK='ext-vm' # (required) The VM network to deploy the management cluster on
export VSPHERE_RESOURCE_POOL='k8s' # (required) The vSphere resource pool for your VMs
export VSPHERE_FOLDER='k8s' # (optional) The VM folder for your VMs, defaults to the root vSphere folder if not set.
export VSPHERE_TEMPLATE='template_ubuntu-1804-kube-v1.16.3' # (required) The VM template to use for your management cluster.
export VSPHERE_DISK_GIB='50' # (optional) The VM Disk size in GB, defaults to 20 if not set
export VSPHERE_NUM_CPUS='2' # (optional) The # of CPUs for control plane nodes in your management cluster, defaults to 2 if not set
export VSPHERE_MEM_MIB='2048' # (optional) The memory (in MiB) for control plane nodes in your management cluster, defaults to 2048 if not set
export SSH_AUTHORIZED_KEY='ssh-rsa AAAAB...6Ix0= kuro@kuro-ubuntu01' # (optional) The public ssh authorized key on all machines in this cluster
# Kubernetes configs
export KUBERNETES_VERSION='1.16.3' # (optional) The Kubernetes version to use, defaults to 1.16.2
export SERVICE_CIDR='100.64.0.0/13' # (optional) The service CIDR of the management cluster, defaults to "100.64.0.0/13"
export CLUSTER_CIDR='100.96.0.0/11' # (optional) The cluster CIDR of the management cluster, defaults to "100.96.0.0/11"
export SERVICE_DOMAIN='cluster.local' # (optional) The k8s service domain of the management cluster, defaults to "cluster.local"
$ ls -l ./out/management-cluster/
total 268
-rw-r--r-- 1 kuro kuro 19656 Feb 11 08:54 addons.yaml
-rw-r--r-- 1 kuro kuro 933 Feb 11 08:54 cluster.yaml
-rw-r--r-- 1 kuro kuro 3649 Feb 11 08:54 controlplane.yaml
-rw-r--r-- 1 kuro kuro 2576 Feb 11 08:54 machinedeployment.yaml
-rw-r--r-- 1 kuro kuro 240747 Feb 11 08:54 provider-components.yaml
で、あとはこれらを clusterctl に食べさせるだけです。
$ clusterctl create cluster \
--bootstrap-type kind \
--bootstrap-flags name=management-cluster \
--cluster ./out/management-cluster/cluster.yaml \
--machines ./out/management-cluster/controlplane.yaml \
--provider-components ./out/management-cluster/provider-components.yaml \
--addon-components ./out/management-cluster/addons.yaml \
--kubeconfig-out ./out/management-cluster/kubeconfig
NOTICE: clusterctl has been deprecated in v1alpha2 and will be removed in a future version.
I0211 08:55:09.414116 2332 createbootstrapcluster.go:27] Preparing bootstrap cluster
I0211 08:55:59.745617 2332 clusterdeployer.go:82] Applying Cluster API stack to bootstrap cluster
...
I0211 08:56:02.302440 2332 clusterdeployer.go:87] Provisioning target cluster via bootstrap cluster
...
I0211 08:56:02.454960 2332 applymachines.go:46] Creating machines in namespace "default"
I0211 08:59:12.512879 2332 clusterdeployer.go:105] Creating target cluster
...
I0211 08:59:13.394821 2332 clusterdeployer.go:123] Pivoting Cluster API stack to target cluster
...
I0211 08:59:56.665878 2332 clusterdeployer.go:164] Done provisioning cluster. You can now access your cluster with kubectl --kubeconfig ./out/management-cluster/kubeconfig
I0211 08:59:56.666586 2332 createbootstrapcluster.go:36] Cleaning up bootstrap cluster.
主要なログだけ抜粋していますが、これだけで、
Kind を使って、作業端末の Docker 上にブートストラップクラスタを構築する
ブートストラップクラスタに Cluster API を導入する
その Cluster API を使って、マニフェスト通りに vSphere 環境上にマネジメントクラスタをデプロイする
仮想マシンをテンプレートからデプロイする
仮想マシンをクラスタのコントロールプレーンとして構成する
ブートストラップクラスタ上に構成していた Cluster API の環境をマネジメントクラスタに移行する（Pivoting）
$ export KUBECONFIG="$(pwd)/out/management-cluster/kubeconfig"
$ kubectl cluster-info
Kubernetes master is running at https://192.168.0.27:6443
KubeDNS is running at https://192.168.0.27:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
$ kubectl get machines
NAME PROVIDERID PHASE
management-cluster-controlplane-0 vsphere://42069f1f-21ac-45be-69b9-e94702d3062b running
$ export KUBECONFIG="$(pwd)/out/management-cluster/kubeconfig"
$ kubectl apply -f ./out/workload-cluster-1/cluster.yaml
cluster.cluster.x-k8s.io/workload-cluster-1 created
vspherecluster.infrastructure.cluster.x-k8s.io/workload-cluster-1 created
$ kubectl apply -f ./out/workload-cluster-1/controlplane.yaml
kubeadmconfig.bootstrap.cluster.x-k8s.io/workload-cluster-1-controlplane-0 created
machine.cluster.x-k8s.io/workload-cluster-1-controlplane-0 created
vspheremachine.infrastructure.cluster.x-k8s.io/workload-cluster-1-controlplane-0 created
$ kubectl apply -f ./out/workload-cluster-1/machinedeployment.yaml
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/workload-cluster-1-md-0 created
machinedeployment.cluster.x-k8s.io/workload-cluster-1-md-0 created
vspheremachinetemplate.infrastructure.cluster.x-k8s.io/workload-cluster-1-md-0 created
$ kubectl get secrets
NAME TYPE DATA AGE
...
workload-cluster-1-kubeconfig Opaque 1
...
$ kubectl get secret workload-cluster-1-kubeconfig -o=jsonpath='{.data.value}' | \
{ base64 -d 2>/dev/null || base64 -D; } >./out/workload-cluster-1/kubeconfig
実際に接続すれば、当たり前ですがゲストクラスタやノードの情報が確認できます。
$ export KUBECONFIG="$(pwd)/out/workload-cluster-1/kubeconfig"
$ kubectl cluster-info
Kubernetes master is running at https://192.168.0.28:6443
KubeDNS is running at https://192.168.0.28:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
workload-cluster-1-controlplane-0 NotReady master 3m14s v1.16.3
workload-cluster-1-md-0-78469c8cf9-fr22j NotReady <none> 90s v1.16.3
$ kubectl apply -f ./out/workload-cluster-1/addons.yaml
configmap/calico-config created
...
serviceaccount/calico-kube-controllers created
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
workload-cluster-1-controlplane-0 Ready master 4m22s v1.16.3
workload-cluster-1-md-0-78469c8cf9-fr22j Ready <none> 2m38s v1.16.3
$ kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.8.3/manifests/metallb.yaml
namespace/metallb-system created
...
deployment.apps/controller created
