Cluster API is still in a prototype stage while we get feedback on the API types themselves. All of the code here is to experiment with the API and demo its abilities, in order to drive more technical feedback to the API design. Because of this, all of the codebase is rapidly changing.
$ cat envvars.txt
# vCenter config/credentials
export VSPHERE_SERVER='192.168.0.201' # (required) The vCenter server IP or FQDN
export VSPHERE_USERNAME='administrator@vsphere.local' # (required) The username used to access the remote vSphere endpoint
export VSPHERE_PASSWORD='my-secure-password' # (required) The password used to access the remote vSphere endpoint
# vSphere deployment configs
export VSPHERE_DATACENTER='kuro-dc01' # (required) The vSphere datacenter to deploy the management cluster on
export VSPHERE_DATASTORE='nfs-ds01' # (required) The vSphere datastore to deploy the management cluster on
export VSPHERE_NETWORK='ext-vm' # (required) The VM network to deploy the management cluster on
export VSPHERE_RESOURCE_POOL='k8s' # (required) The vSphere resource pool for your VMs
export VSPHERE_FOLDER='k8s' # (optional) The VM folder for your VMs, defaults to the root vSphere folder if not set.
export VSPHERE_TEMPLATE='template_ubuntu-1804-kube-v1.16.3' # (required) The VM template to use for your management cluster.
export VSPHERE_DISK_GIB='50' # (optional) The VM Disk size in GB, defaults to 20 if not set
export VSPHERE_NUM_CPUS='2' # (optional) The # of CPUs for control plane nodes in your management cluster, defaults to 2 if not set
export VSPHERE_MEM_MIB='2048' # (optional) The memory (in MiB) for control plane nodes in your management cluster, defaults to 2048 if not set
export SSH_AUTHORIZED_KEY='ssh-rsa AAAAB...6Ix0= kuro@kuro-ubuntu01' # (optional) The public ssh authorized key on all machines in this cluster
# Kubernetes configs
export KUBERNETES_VERSION='1.16.3' # (optional) The Kubernetes version to use, defaults to 1.16.2
export SERVICE_CIDR='100.64.0.0/13' # (optional) The service CIDR of the management cluster, defaults to "100.64.0.0/13"
export CLUSTER_CIDR='100.96.0.0/11' # (optional) The cluster CIDR of the management cluster, defaults to "100.96.0.0/11"
export SERVICE_DOMAIN='cluster.local' # (optional) The k8s service domain of the management cluster, defaults to "cluster.local"
$ ls -l ./out/management-cluster/
total 268
-rw-r--r-- 1 kuro kuro 19656 Feb 11 08:54 addons.yaml
-rw-r--r-- 1 kuro kuro 933 Feb 11 08:54 cluster.yaml
-rw-r--r-- 1 kuro kuro 3649 Feb 11 08:54 controlplane.yaml
-rw-r--r-- 1 kuro kuro 2576 Feb 11 08:54 machinedeployment.yaml
-rw-r--r-- 1 kuro kuro 240747 Feb 11 08:54 provider-components.yaml
で、あとはこれらを clusterctl に食べさせるだけです。
$ clusterctl create cluster \
--bootstrap-type kind \
--bootstrap-flags name=management-cluster \
--cluster ./out/management-cluster/cluster.yaml \
--machines ./out/management-cluster/controlplane.yaml \
--provider-components ./out/management-cluster/provider-components.yaml \
--addon-components ./out/management-cluster/addons.yaml \
--kubeconfig-out ./out/management-cluster/kubeconfig
NOTICE: clusterctl has been deprecated in v1alpha2 and will be removed in a future version.
I0211 08:55:09.414116 2332 createbootstrapcluster.go:27] Preparing bootstrap cluster
I0211 08:55:59.745617 2332 clusterdeployer.go:82] Applying Cluster API stack to bootstrap cluster
...
I0211 08:56:02.302440 2332 clusterdeployer.go:87] Provisioning target cluster via bootstrap cluster
...
I0211 08:56:02.454960 2332 applymachines.go:46] Creating machines in namespace "default"
I0211 08:59:12.512879 2332 clusterdeployer.go:105] Creating target cluster
...
I0211 08:59:13.394821 2332 clusterdeployer.go:123] Pivoting Cluster API stack to target cluster
...
I0211 08:59:56.665878 2332 clusterdeployer.go:164] Done provisioning cluster. You can now access your cluster with kubectl --kubeconfig ./out/management-cluster/kubeconfig
I0211 08:59:56.666586 2332 createbootstrapcluster.go:36] Cleaning up bootstrap cluster.
主要なログだけ抜粋していますが、これだけで、
Kind を使って、作業端末の Docker 上にブートストラップクラスタを構築する
ブートストラップクラスタに Cluster API を導入する
その Cluster API を使って、マニフェスト通りに vSphere 環境上にマネジメントクラスタをデプロイする
仮想マシンをテンプレートからデプロイする
仮想マシンをクラスタのコントロールプレーンとして構成する
ブートストラップクラスタ上に構成していた Cluster API の環境をマネジメントクラスタに移行する（Pivoting）
$ export KUBECONFIG="$(pwd)/out/management-cluster/kubeconfig"
$ kubectl cluster-info
Kubernetes master is running at https://192.168.0.27:6443
KubeDNS is running at https://192.168.0.27:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
$ kubectl get machines
NAME PROVIDERID PHASE
management-cluster-controlplane-0 vsphere://42069f1f-21ac-45be-69b9-e94702d3062b running
$ export KUBECONFIG="$(pwd)/out/management-cluster/kubeconfig"
$ kubectl apply -f ./out/workload-cluster-1/cluster.yaml
cluster.cluster.x-k8s.io/workload-cluster-1 created
vspherecluster.infrastructure.cluster.x-k8s.io/workload-cluster-1 created
$ kubectl apply -f ./out/workload-cluster-1/controlplane.yaml
kubeadmconfig.bootstrap.cluster.x-k8s.io/workload-cluster-1-controlplane-0 created
machine.cluster.x-k8s.io/workload-cluster-1-controlplane-0 created
vspheremachine.infrastructure.cluster.x-k8s.io/workload-cluster-1-controlplane-0 created
$ kubectl apply -f ./out/workload-cluster-1/machinedeployment.yaml
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/workload-cluster-1-md-0 created
machinedeployment.cluster.x-k8s.io/workload-cluster-1-md-0 created
vspheremachinetemplate.infrastructure.cluster.x-k8s.io/workload-cluster-1-md-0 created
$ kubectl get secrets
NAME TYPE DATA AGE
...
workload-cluster-1-kubeconfig Opaque 1
...
$ kubectl get secret workload-cluster-1-kubeconfig -o=jsonpath='{.data.value}' | \
{ base64 -d 2>/dev/null || base64 -D; } >./out/workload-cluster-1/kubeconfig
実際に接続すれば、当たり前ですがゲストクラスタやノードの情報が確認できます。
$ export KUBECONFIG="$(pwd)/out/workload-cluster-1/kubeconfig"
$ kubectl cluster-info
Kubernetes master is running at https://192.168.0.28:6443
KubeDNS is running at https://192.168.0.28:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
$ kubectl get podsnodes
NAME STATUS ROLES AGE VERSION
workload-cluster-1-controlplane-0 NotReady master 3m14s v1.16.3
workload-cluster-1-md-0-78469c8cf9-fr22j NotReady <none> 90s v1.16.3
$ kubectl apply -f ./out/workload-cluster-1/addons.yaml
configmap/calico-config created
...
serviceaccount/calico-kube-controllers created
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
workload-cluster-1-controlplane-0 Ready master 4m22s v1.16.3
workload-cluster-1-md-0-78469c8cf9-fr22j Ready <none> 2m38s v1.16.3
$ kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.8.3/manifests/metallb.yaml
namespace/metallb-system created
...
deployment.apps/controller created
のいずれかの条件を満たす必要がありますが、加湿器でこれに合致するモデルが全然見つけられませんでした。大体の加湿器は、コンセントの通電後にさらにボタンを押さないと動作しませんし、ボタンの周辺の形状や寸法やボタンの硬さは公開されていませんし、リモコン付きのモデルでも赤外線でなく RF ですし、公式の API はなさそうですし。
エアコンを操作する Nature Remo mini と加湿器
結局、公開されていない API を無理やり叩ける非公式のライブラリが存在していてハックの余地がありそう、ということがわかった Oittm のアロマディフューザー を採用しています。ただしこれも、
$ docker run -d --rm --name broker -p 1883:1883 eclipse-mosquitto
$ docker run --init --rm -it efrecon/mqtt-client sub -h 192.168.0.100 -t "#" -v
別のターミナルで REST エンドポイントも起動します。
$ cd rest-endpoint
$ python ./main.py
* Serving Flask app "main" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Restarting with stat
* Debugger is active!
* Debugger PIN: 159-538-312
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
$ docker run -d --rm --name broker -p 1883:1883 eclipse-mosquitto
$ docker run -d --restart=always --name=mqtt-scripts -v "$(pwd)/mqtt-scripts:/scripts" dersimn/mqtt-scripts --url mqtt://192.168.0.100 --dir /scripts
$ docker-compose up -d
REST デバイスサービスの動作確認
起動が確認できたら、実際にエンドポイントに POST して、データの取り込みを確認します。待ち受けポートは 49986 です。Content-Type は text/plain である必要があります。
$ curl -X POST -H "Content-Type: text/plain" -d 12345 http://localhost:49986/api/v1/resource/REST_DEVICE/int
edgex-device-rest コンテナのログで Pushed event to core data が記録されていれば、値は正常に受け付けられています。
$ docker logs --tail 5 edgex-device-rest
level=DEBUG ts=2020-01-25T07:49:17.0621062Z app=edgex-device-rest source=resthandler.go:82 msg="Received POST for Device=REST_DEVICE Resource=int"
level=DEBUG ts=2020-01-25T07:49:17.0673842Z app=edgex-device-rest source=resthandler.go:101 msg="Content Type is 'text/plain' & Media Type is 'text/plain' and Type is 'Int64'"
level=DEBUG ts=2020-01-25T07:49:17.0726606Z app=edgex-device-rest source=resthandler.go:142 msg="Incoming reading received: Device=REST_DEVICE Resource=int"
level=DEBUG ts=2020-01-25T07:49:17.0766349Z app=edgex-device-rest source=utils.go:75 correlation-id=946f482c-db1f-44b5-8a76-437c4ca3d3e9 msg="SendEvent: EventClient.MarshalEvent encoded event"
level=INFO ts=2020-01-25T07:49:17.0899243Z app=edgex-device-rest source=utils.go:93 Content-Type=application/json correlation-id=946f482c-db1f-44b5-8a76-437c4ca3d3e9 msg="SendEvent: Pushed event to core data"
実際に取り込まれているようです。
$ edgex-cli reading list REST_DEVICE -l 10
Reading ID Name Device Origin Value Created Modified Pushed
b43f7300-1377-4fdb-a82f-44eb497d9568 int REST_DEVICE 1579938557072646900 12345 3 minutes 3 minutes 50 years
$ docker exec edgex-support-rulesengine cat /edgex/edgex-support-rulesengine/rules/rule_int_high.drl
package org.edgexfoundry.rules;
global org.edgexfoundry.engine.CommandExecutor executor;
global org.edgexfoundry.support.logging.client.EdgeXLogger logger;
import org.edgexfoundry.domain.core.Event;
import org.edgexfoundry.domain.core.Reading;
import java.util.Map;
rule "rule_int_high"
when
$e:Event($rlist: readings && device=="REST_DEVICE")
$r0:Reading(name=="int" && Integer.parseInt(value) > 80) from $rlist
then
executor.fireCommand("09dae1fc-e2be-4388-9677-639d2f24c58b", "1c7a50e7-5424-4bce-9b9a-29849510580e", "{\"message\":\"HIGH\"}");
logger.info("Action triggered: The value is too high.");
end
$ docker exec edgex-support-rulesengine cat /edgex/edgex-support-rulesengine/rules/rule_int_low.drl
package org.edgexfoundry.rules;
global org.edgexfoundry.engine.CommandExecutor executor;
global org.edgexfoundry.support.logging.client.EdgeXLogger logger;
import org.edgexfoundry.domain.core.Event;
import org.edgexfoundry.domain.core.Reading;
import java.util.Map;
rule "rule_int_low"
when
$e:Event($rlist: readings && device=="REST_DEVICE")
$r0:Reading(name=="int" && Integer.parseInt(value) < 20) from $rlist
then
executor.fireCommand("09dae1fc-e2be-4388-9677-639d2f24c58b", "1c7a50e7-5424-4bce-9b9a-29849510580e", "{\"message\":\"LOW\"}");
logger.info("Action triggered: The value is too low.");
end
$ docker logs -f --tail=10 edgex-support-rulesengine
[2020-01-25 13:44:15.023] boot - 6 INFO [main] --- ZeroMQEventSubscriber: JSON event received
[2020-01-25 13:44:15.024] boot - 6 INFO [main] --- ZeroMQEventSubscriber: Event sent to rules engine for device id: MQ_DEVICE
[2020-01-25 13:44:17.214] boot - 6 INFO [main] --- ZeroMQEventSubscriber: JSON event received
[2020-01-25 13:44:17.216] boot - 6 INFO [main] --- ZeroMQEventSubscriber: Event sent to rules engine for device id: MQ_DEVICE
...
$ docker run --init --rm -it efrecon/mqtt-client sub -h 192.168.0.100 -t "#" -v
logic/connected 2
DataTopic {"name":"MQ_DEVICE","cmd":"randfloat32","randfloat32":"27.0"}
...
では、REST デバイスの気持ちになって、まずはルールに合致しないデータを REST デバイスサービスに POST します。
$ curl -X POST -H "Content-Type: text/plain" -d 50 http://localhost:49986/api/v1/resource/REST_DEVICE/int
値は取り込まれていますし、
$ edgex-cli reading list REST_DEVICE -l 10
Reading ID Name Device Origin Value Created Modified Pushed
66461804-a894-470f-a099-631ffd4c32cb int REST_DEVICE 1579960045357298000 50 About a minute About a minute 50 years
