Secure Components with Cellery

Cellery is a novel code first approach for building, integrating, running and managing composite applications on Kubernetes. While using your existing container images, you can achieve many more things on top of them with minimum effort. Refer Cellery official website for more information about Cellery

Cellery provides a mechanism of enforcing network boundaries across a set of micro-services which act as a Cell. These Cells have a single entry point which we call as the Cell Gateway. Individual teams can develop, manage, scale, grow and work independently on Cells.

One of the primary design aspects of Cellery is Secure Components. Considering all aspects of security while developing micro services is an overhead to the developer. With Cellery, developers don’t need to bother about security aspects of the services they are writing. Everything is taken cared by the Cellery Runtime itself.

Think about a typical API management scenario where you need to expose a secured API through an API Gateway. This includes a number of manual tasks for a traditional kubernetes deployment where as in Cellery, you can define all these requirements from the language syntax itself and deploy the artifact into Cellery runtime.

What about secured APIs ? Well, in Cellery the APIs you publish into global gateway are secured by default through OAuth2. You need to have a token received through a valid subscription in order to invoke those APIs. We call this as Edge security where we validate the end user tokens and allow the user to enter to the mesh.

Let’s diverge from edge security and dig more into Cell security as it’s one of the most important things about Cellery. Being independent teams which manage independent Cells, developers find hard to enforce access control and apply security policies for their services with the scattered nature and complex use cases. Developers in one team don’t have control over other Cells which do not belong to their team. Hence having a concrete way of controlling access and enforcing access control through a single point is very important in Cell perspective.

How does Cellery solve this problem ?

Think about independent trust domains/entities which try to bootstrap trust and communicate through a secured way. The important problem is how can an entity prove the source and the destination of a request and how entities can securely convey the information in a tamper proof manner. This is a standard problem when it comes to monolithic IAM systems. Aforementioned problem becomes challenging when it comes to micro-services world where services spawn and vanish in a dynamic way.

Cellery models Cells as independent trust domains. Each Cell has a key pair which is signed and issued by the Cellery Controller. Controller is responsible for bootstrapping trust between Cells by issuing above mentioned key pairs. Each Cell has a Secure Token Service (STS) which is responsible for enforcing security and access control for a Cell. This is the centralized place where Cell security is taken cared at. In other terms the Controller signed key pair is issued to each Cell STS, so that STS can sign and issue assertions towards workloads invoking services exposed by other workloads.

Cells can dynamically spawn and terminate as same as micro services. An independent Cell doesn’t know about the existence of other cells. Hence the design should support a mechanism to dynamically discover what other cells are and what their keys are in order to verify assertions. Cellery uses JWKS endpoints which are exposed by each Cell STS to expose their public keys to outside. An entity which needs to verify an assertion issued by a Cell STS can talk to JWKS endpoint of the respective Cell and verify signature by retrieving keys. All JWKS endpoints have their unique service name which has Cell name as a part of it and are exposed through HTTPS using the keys distributed by Controller. Hence the trust is bootstrapped within Cells through this mechanism.

Let’s walk through a practical scenario.

If you take the HR sample in Cellery samples, this is a simple micro service which is exposed through global gateway with OAuth2 protection.

In order to run this Cell, save above content in a file with name hello-world-api.bal and run the below command. Follow the guid in the sample and deploy the cell. Try with the below curl

curl https://wso2-apim-gateway/hello-world-api-cell/hello/ -k

This will back an error with status code 403 (forbidden)

This is because the API which is exposed globally is protected with OAuth2. If you edit the cell definition file and remove authenticate: false part and then build and redeploy, you should be able to access the API without any token.

Above is some insight towards Edge Security. When considering about inter cell security, the topic becomes more broad and complex. A service which is lying inside a cell needs to have a token issued by the resident Cells’ STS in order to invoke another service which relies in another Cell. This is the simplest form of inter Cell security you can have.

Let’s do some hands on work, to demonstrate how intercell security work. First you have to deploy the above hello world Cell, with security enabled. ie you need to have a valid token in order to access it through both global gateway and through kubernetes service mesh. Let’s try to deploy a bogus pod in kubernetes mesh and try to invoke hello-world api directly through that bogus pod. Please note that we are not trying to access the service through global gateway. Instead we are trying to access it directly through kubernetes service which is only exposed inside mesh.

First install a bogus pod by running the below

kubectl apply -f https://raw.githubusercontent.com/Mirage20/k8s-debug-tools/master/pod.yaml

List pods and identify the newly installed debug pod

kubectl get pods
NAME READY STATUS RESTARTS AGE
debug-tools 1/1 Running 0 2m11s
hello-world-api-cell — gateway-deployment-579d47698d-hqv59 2/2 Running 0 17m
hello-world-api-cell — hello-api-deployment-fc696d79b-fc97f 2/2 Running 0 17m
hello-world-api-cell — sts-deployment-56c55b8488-g6mz2 3/3 Running 0 17m

Exec into pod so that you can run commands inside it

kubectl exec -it debug-tools /bin/bash

Try to access the kubernetes service through bogus pod by executing the below curl

curl -k -v http://hello-world-api-cell--gateway-service:80/hello

You will get the below response.

Trying 10.109.125.46…
* Connected to hello-world-api-cell — gateway-service (10.109.125.46) port 80 (#0)
> GET /hello HTTP/1.1
> Host: hello-world-api-cell — gateway-service
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 403 Forbidden
< date: Tue, 13 Aug 2019 10:19:43 GMT
< server: envoy
< x-envoy-decorator-operation: hello-world-api-cell — gateway-service.default.svc.cluster.local:80/*
< content-length: 0
<

The response status code is 403 which stands for Forbidden. This is the simplest form of intercell security components can have. A component which is does not belong to any verified cell cannot access services in any terms.

Conclusion

Cellery is designed to cater more complex security requirements. Starting from Cell security, Cellery is designed in a way which empowers the developer to gain complex and advance security requirements with minimum effort. You can control access to each service while enforcing fine grained access control based on any complex conditions such as user attributes, environment attributes, request source and destination and many more. Having OPA engine embedded inside Cellery Runtime makes your security policies configurable and deployable as you wish.

In the next article we will discuss how to achieve your advanced security requirements using Cellery with the help of OPA policy engine which is quite interesting.