Simple Baremetal Kubernetes Dashboard with Ingress

Hi, in this article I will share my baremetal experiments on Kubernetes and try to explain the parts(resources) that I used to figure out to reach Kubernetes Dashboard service with a fake domain name in a local network. As you know, to use fake domains we need to add the fake domain name into the hosts file :) but in which machine(node) ? :)

I would like to say about Ingress very briefly that it is like a server which serves Kubernetes cluster to the outside, reverse proxy. Let’s think about you have web site folders, web pages, scripts, so on and you need a server, even to serve these web site contents on “localhost” environment. Even, by utilizing server properties you can serve a file by a different url that is different than default server url routing mappings. For example, let’s say there is a html file and it’s default url maybe ‘localhost/file_1.html” but you can serve this file by creating this simple url ‘localhost/my_fav_first_file” by utilizing url rewrite specifications. By Ingress, you can make your manipulations in your Kubernetes environment.

In your local environment experiments it will be very useful to use port forwarding to reach kubernetes services. You need to know some details to figure out in which machine(node) to create port forwarding. In kubernetes you can think pods as virtual machines in nodes. We need this clue to create port forwarding for ingress service which is ingress-controller. We need to locate where ingress-controller service is in. For instance we may have three machines(nodes) and their IP addresses may be likae that, 192.168.1.33(master node), 192.168.1.35(worker-1) and 192.168.1.42(worker-2), and let’s say our ingress-controller app is living in worker-1 machine. I will also share how to create ingress-controller service a little bit later in this article.

As you know, services are another abstraction of the deployed applications since they can have multiple replications on different pods. You can think of services as reverse proxy server which has service name such as domain name and a IP. It forwards incoming requests to related pods where the related application is living and pods also have IP addresses. Services are reverse proxy of related pod or pods. By replication, the applications become more resilient against to errors and many other failures, since if one replica dies there will be many others. Kubernetes manages all of it. Also, for replica perspective there are two differen aspect that Kubernetes can manage them as sateless or stateful. In stateful perspective, kubernetes replicaset creates or deletes the replicas in an order, since if you need, some applications select some of their replica as a master replica and this master replica should be created first and deleted at last. Also for stateful prespective the naming of the pods are in an ordered fashion. You can search the details in the web.

Previously we have said that services are reverse proxies of related pods, so why is then we need ingress reverse proxy service ? Moreover, in kubernetes a service has three different types. LoadBalancer, NodePort and ClusterIP. Ingress-controller service initially is installed as LoadBalancer service. However, in order to use LoadBalancer type you will need real domain address. For our local environment example this type is useless but it does not affect our experiment also. So why do we need ingress ? Since, a service is related with only one application and we expose an application to outher world as a service, SaaS. Kubernetes adds more functionality to the services by design, that’s all. If we want route mapping customizations for services, url rewrite properties on top of the kubernetes cluster, we need ingress like reverse proxy service. Even we can create a service similar to ingress controller service but there is already built one. In order to understand these details more clear, this official kubernetes link will be very helpful: https://kubernetes.io/docs/concepts/services-networking/

Ingress has two main components, ingress-controller service and ingress configuration files. It is good that configuration file is seperated from service part, since, previously has been said that a service can manage multiple replicated pods where the related app(ingress) is living in. All ingress apps(exposed as ingress controller service) read the configuration from ingress configuration resource. There are also configMap resources in kubernetes and by this configmaps, services can reach and get the necessary configurations, but that is a custom work to do so, you should write your application to read configurations from configMaps. On the other hand ingress comes with it’s standard aspect and it simplifies our effort. Ingress controller service(s) will know where to read ingress configurations.

In this part of the article and beyond it is assumed that we have a ready kubernetes cluster which consists at least two nodes, one is master, the other one is worker. Moreover, we should have installed Container Network Interface(CNI) plugin for kubernetes, such as flannel, calico, etc.

There are different ingress controller implementations, one of them can be used and in the examples nginx-ingress will be used. For Ingress controller implementations you can look at that official link: https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/

Now on by this installation yaml, nginx ingress controller will be installed into the kubernetes cluster:
“sudo kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.5.1/deploy/static/provider/cloud/deploy.yaml”

For quick start details you can review this useful link: https://kubernetes.github.io/ingress-nginx/deploy/#quick-start

Before ingress configuration resource installation we need kubernetes dashboard to be installed and by this url directive yaml:

sudo kubectl apply -f 
https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.1/aio/deploy/recommended.yaml

The official link of dashboard is: https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/

Initially the dashboard will be installed as ClusterIP service and you can only reach the dashboard from the node where the dashboard app has been installed. In the node, if somewhat you can use browser and you can type the dahsboard service clusterIP, for example, https://clusterIP. Or you can utilize cli network tools. We can change the configuration of the dashboard service and make that service as NodePort service and by doing so we can reach dashboard from any machine in your local network but you will need to add port value as well. For example, https://192.168.1.33:32576. The ip address is the node IP where dashboard service’s related pod is living ( it is not pod IP adress, it is node IP address). By ingress example we will omit the port value, also we will use fake domain address instead of IP address and ingress will add the port value by url rewrite process. By this brief explanations it may be also understood the difference between ClusterIP and NodePort type of services.

Now we can execute ingress configuration directive yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:   
     nginx.ingress.kubernetes.io/server-snippet: |
      location ~ / {
         proxy_pass https://demo.dashboard.me:32576;
         proxy_set_header Host $host;
         proxy_set_header X-Forwarded-Proto https;
      }
  name: nginx1-ingress
  namespace: kubernetes-dashboard
spec:
  rules:
  - host: demo.dashboard.me
    http:
      paths:
      - backend:
          service:
            name: kubernetes-dashboard
            port:
              number: 80
        path: /
        pathType: Prefix
status:
  loadBalancer: {}

As you may notice that ingress-controller can manage all incoming requests under this url, demo.dashboard.me and rewriting the domain url by adding port number, 32576. Path value ‘/’ means the domain name part of the url so in location attribute this value is mentioned to be replaced as proxy_pass value. By proxy_pass and other proxy attributes, url redirection url value is not changed in the browser. However there will be dashboard login page redirection in our example but port number will never be seen. Also as you may notice, the given port number is the port number of the node machine in which the related dashboard pod is living in.

Now we should create port forwarding for our ingress controller service. You can create port forwading in any cluster machine, that can be master node or worker node. However we will need this machine’s local network IP address (e.g. 192.168.1.33). We add this IP address into our client machine (hosts file) in which we can try this address http://demo.dashboard.me.

Port forwarding command:

sudo kubectl port-forward --address 0.0.0.0  \
--namespace=ingress-nginx service/ingress-nginx-controller 80:80

Also we need another port forwarding for 443 in the same node:

sudo kubectl port-forward --address 0.0.0.0  \
--namespace=ingress-nginx service/ingress-nginx-controller 443:443

We need 443 also, since dashboard service need tls communication and redirections are done under this TLS communication.

If a request comes to that node’s port 80 then this request is forwarded to port 80 of kubernetes cluster and this port is used by ingress controller service. It is same for 443. It could be 8080:80, 9091:80 or any ****:80. We used 80 specifically so that we will not mention any port number in our example url. As you know left part from the semicolon is our machine’s port number, right part is cluster’s network port. Also you can check which application is using these ports in the node, for example for linux machines: sudo lsof -i:80
and the result will be similar to this: kubectl 1432 root 8u IPv4 72945710 0t0 TCP *:http(LISTEN)

If we remove the annotations part in the Ingress yaml file then we will have to write this url to access dashboard: http://demo.dashboard.me:32576. As you can see, without proxy settings we will have to add port number.

Moreover, in the browser you will see untrusted site warning, since we have not implemented any signed certificate management for our dashboard service, especially for our fake domain name. It can be managed also and by this useful youtube video you can try it: https://www.youtube.com/watch?v=IQ3G8Z1myMw

You can learn more about what is forward/reverse proxy server by this youtube link: https://www.youtube.com/watch?v=4NB0NDtOwIQ

For more networking information you can look at this useful youtube channel: https://www.youtube.com/@PowerCertAnimatedVideos

Also I utilized these links to figure out to create this simple example in local environment:

How to specify Proxy Pass in kubernetes

Kubernetes ingress-nginx rewrite without changing browser url

Rewrite URL in Kubernetes with Ingress

How to rewrite url to backend while preserving browser url in nginx ingress?

Thanks for reading,
Mustafa Cem Yaşar