Kubernetes: Local Private Registry

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
 
## Setup local private registry



- After the experiment is fully deployed, SSH into the head node and 
run the following:

~~~bash
$ bash /local/repository/registry/setup_registry.sh
$ cp /local/repository/registry/docker-compose.yml .
$ docker-compose up -d
$ docker container ps
~~~



- SSH to **each node** of the experiment and run the followings

~~~bash
$ sudo cp -R /opt/keys/certs.d /etc/docker/
~~~

- Get the IP address from the head node using the following command

~~~bash
$ ip addr | grep eth0$ | awk -F ' ' '{print $2}' | awk -F '/' '{print $1'}
~~~

- On each node, test that the certificates are working by login 
to the local docker registry using the following command
  - You should see a "Login Suceeded" message. 

~~~bash
$ docker login -u admin -p registry https://IP_ADDRESS_FROM_ABOVE:443
~~~



- On the head node, run the following
  - **Assume that you already run docker login to the private local registry on head node**
  - If your app is deployed inside a namespace, the `regcred` generic secret 
  creation command needs to be specified with that specific namespace as well. 

~~~bash
$ ip_addr=$(ip addr | grep eth0$ | awk -F ' ' '{print $2}' | awk -F '/' '{print $1'})
$ kubectl create secret generic registry-ca --namespace kube-system --from-file=registry-ca=/opt/keys/certs.d/${ip_addr}\:443/ca.crt
$ kubectl create -f registry-ca-ds.yaml
$ kubectl create secret generic regcred --from-file=.dockerconfigjson=/users/${USER}/.docker/config.json --type=kubernetes.io/dockerconfigjson
~~~

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
 
## Launch pipelines


- Fork the [`hello` repository](https://github.com/CSC603-WCU/hello) to your Git account
- Setup the API Token for `pikachu`:
  - On Jenkins main page, Go to `People`, then select `pokemon`, then go to `Configure`.
  - Scroll to `API Token`, `Add new Token` and generate one. You can name it `github`. 
  - Copy and store this token for today's class (`JENKINS_TOKEN`). 
- Setup the `webhook` for your repository. 
  - Go to your Git repository's `Settings`, and scroll to `Webhooks` 
  - Add a new webhook with the following configurations:
    - The Payload URL is of the following format: `http://pokemon:JENKINS_TOKEN@IP_ADDRESS:30000/github-webhook/` 
    - The Content type is `application/x-www-form-urlencoded`
- The composition of the files in the `go_app` branch includes:
  - `main.go`: The Go file that serves as the web server (the application to be deployed).
  - `main_test.go`: The Go file that serves as the test file (part of the CD process).
  - `Jenkinsfile`: Setup the pipeline for Jenkins to build, test, and push and deploy (if test is passed) the Go app. 
    - Edit the `registry` (line 6) to be the IP address of the `head` node. 
    - Change the `userid` value from `lngo` to your CloudLab username. 
    - **Be careful of capitalization in your CloudLab username. It has to match exactly**.
  - `Dockerfile`: The Docker image that will package the web server. 
  - `deployment.yml` and `service.yml`: K8 configuration files. 



- Login to the Jenkins server. 
- Select `New Item`, and create a new `Pipeline`named `go_server`.
- Scroll down to `Build Triggers`, select `GitHub hook trigger for GITScm polling`,
- Scroll down to `Pipeline`, select the followings: 
  - `Definition`: Pipeline script from SCM (*this will open new options*)
  - `Repository URL`: URL of the `hello` repository
  - `SCM`: Git
  - `Branches to build`: `go_app` 
- Click `Save`
- Click `Build Now` to activate the first build
- Open a new browser tab and visit the IP address of `head` at port 32000 to see the running server


- Edit `main.go` in `go_app` to introduce an error.
- Observe that the build failed, but the web server is still running. 
- Change `main.go` and also `main_test.go` so that the build and test can pass. 
- Observe the webserver updated after the build completes successfully.