SuperMap iServer Data Science Services provides a distribution package in the form of a "Helm Chart" for rapid deployment of iServer Data Science Services in Kubernetes (K8S) clusters. Data science service runs in K8S cluster, Notebook of each user Server runs independently in a K8S Pod, which supports concurrent access by a large number of users without interference, and better realizes user-level quarantine.
Helm Chart, a SuperMap iServer data science service, provides a standalone distribution package and is hosted in a third-party Chart repository. Domestic Repository Address: https://isupermap.gitee.io/helm-charts。
Environmental requirements for deploying SuprMap iServer Data Science Services in the form of a Helm Chart, including:
After the above environment is prepared, the following steps will describe how to install and configure iServer Data Science Service Helm in the K8S cluster Chart。
Helm is a package management tool for K8S to simplify the deployment and management of K8S applications.
Take Helm v3.1.2 as an example, download Linux amd64 version:
wget https://get.helm.sh/helm-v3.1.2-linux-amd64.tar.gz
tar -zxvf helm-v3.1.2-linux-amd64.tar.gz
cp linux-amd64/helm /usr/local/bin/
Add Helm Chart Warehouse
helm repo add isupermap http://isupermap.gitee.io/helm-charts
helm repo update # Update Local Index
Install and start iServer Data Science Services
NAMESPACE=supermap
kubectl create namespace $NAMESPACE # create namespace
helm install iserver-datascience isupermap/iserver-datascience -n $NAMESPACE # install and start
The above command is to install and start the iServer Data Science Service in the default configuration, and then configure SuperMap iServer Data Science Services Helm Chart section lists the parameters that can be configured prior to installation.
At this point, the iServer Data Science Service startup is complete.
You can verify that the iServer Data Science Service has started by using the following command:
NAMESPACE=supermap
kubectl get pod -n $NAMESPACE
The following message indicates that the iServer Data Science Service startup is complete.
Enter the http://<K8S Cluster Node IP >:31000 in the browser to access the iServer Data Science Services.
Please refer to IServer Data Science Service User Manual .
Uninstall the iServer Data Science Service Helm Chart using the following command:
NAMESPACE=supermap
helm3 uninstall iserver-datascience --namespace $NAMESPACE
The configurable parameters of the SuperMap iServer Data Science Service Helm Chart and its Description can be found in the As seen in the values.yaml. The following table lists the configurable parameters for the iServer Data Science Service Helm Chart and their default values.
| Parameter | Description | Default value |
| hub.db.type | Configure how the iServer Data Science Service is stored, either emptyDir or pvc | emptyDir |
| hub.db.pvc | Takes effect when the hub.db.type value is pvc | > </</td> |
| hub.db.pvc.storageClassName | The storage Class name used by PVC | > </</td> |
| hub.db.pvc.storage | Size of storage space | 1Gi |
| hub.db.pvc.accessModes | Mount the PV access type | [ReadWriteOnce] |
| hub.image | IServer Data Science Services mirror information, including the mirror name and label. You can change this parameter if you need to change the version of the iServer Data Science Services image | {name: registry.cn-beijing.aliyuncs.com/supermap/iserver-datascience, tag: latest } |
| hub.imagePullPolicy | IServer Data Science Services Mirror Pull Policy | IfNotPresent |
| proxy.secretToken | A 32-byte encrypted secure randomly-generated string used to secure communication between the hub and a configurable HTTP proxy. The default value can be used to run normally, and it is recommended to modify it for safety. | "Fixed random value" |
| proxy.service | K8S exposes an object for the IserverData science service | > </</td> |
| proxy.service.type | K8S service type | LoadBalancer |
| proxy.service.nodePorts | K8S service nodePorts passes through the cluster nodes outside the cluster when the type is nodePort or LoadBalancer ip: nodePort can access the service | {http:31000} |
| proxy.service.loadBalancerIP | Effective when proxy.service.type is LoadBalancer | |
| singleuser.storage.type | Notebook user's Data Save mounting mode is not mounted by default. If pvc mounting is required, static or dynamic can be selected. | > </</td> |
| singleuser.storage.static | Static Mount. Takes effect when the singleuser.storage.type value is static. All user data is mounted using a single pvc, which is provided by the user. | {pvcName:"",subPath: '{username}'} |
| singleuser.storage.dynamic | Dynamic mount, which takes effect when the singleuser. Storage. Type value is dynamic, when a user logs on, using the configured K8S storage class automatically creates PV mount, and data is quarantine between users. | {storageClass:"",pvcNameTemplate: claim-{username}{servername},volumeNameTemplate: volume-{username}{servername},storageAccessModes: [ReadWriteOnce]} |
| singleuser.storage.extraVolumes | Additional volumes, added in K8S format | [] |
| singleuser.storage.extraVolumeMounts | Mount additional volumes, added in K8S format, to /opt/datascience/notebooks/data as a shared directory | [] |
| singleuser.image | Images used by Notebook | { name: registry.cn-beijing.aliyuncs.com/supermap/iserver-datascience, tag: latest , pullPolicy: IfNotPresent} |
| singleuser.extra_container_config.ports | Notebook container opens an additional port to the cluster, which is Spark open port by default | {containerPort: 31234},{containerPort: 31235},{containerPort: 31236},{containerPort: 4040} |
| singleuser.service.type | Notebook service type | LoadBalancer |
| singleuser.service.driverport | An open port is required when using the Spark cluster, and the port number needs to be consistent with the external Spark cluster port number set in the Notebook | {port: 31001,nodePort: 31001} |
| singleuser.service.blockport | An open port is required when using the Spark cluster, and the port number needs to be consistent with the external Spark cluster port number set in the Notebook | {port: 31002,nodePort: 31002} |
| singleuser.service.executorport | An open port is required when using the Spark cluster, and the port number needs to be consistent with the external Spark cluster port number set in the Notebook | {port: 31003,nodePort: 31003} |
| singleuser.service.workui | Open ports are required when using a Spark cluster | {port: 4040,nodePort: 31004} |
You can Modify the above parameters in the following two ways before installing iServer Data Science Service Helm Chart:
Method 1: Configure parameters by creating a new config.yaml Config File.
For example, set the storage mode of iServer Data Science Service Helm Chart as storageClass of PVC, and dynamically mount Data Save for Notebook users. The Data Saves of different users are in quarantine with each other. Reference examples are as follows:
hub:
db:
type: pvc
pvc:
storageClassName: managed-nfs-storage
singleuser:
extra_container_config:
storage:
type: dynamic
storageClass: managed-nfs-storage
Note: The above example requires you to set up your own NFS and create a storageClass.
NAMESPACE=supermap
helm install iserver-datascience isupermap/iserver-datascience -n $NAMESPACE -f config.yaml
Method 2: Configure the parameters by --set key = value [, key = value]
For example, to set a proxy.secretToken security token, you can execute the following command to generate a Token:
openssl rand -hex 32
Copy it, set up the proxy.secretToken security token with the following command, and install and start the iServer Data Science Services
NAMESPACE=supermap
helm install iserver-datascience isupermap/iserver-datascience -n $NAMESPACE --set proxy.secretToken="acfde3dd7231d43ac1a246171c72c2afc7cdb413e5cc87bf611ba9179e19fa6a"