Interlock for Docker Swarm

In early 2018 Docker made an announcement of the release of its newest Docker Enterprise 2.0 product.  This newest release provides a significant advancement in the Docker platform in the form of a choice between Swarm and/or Kubernetes orchestration. But that’s not what i want to talk about.

Layer 7 Routing

Another great addition to the platform is the replacement of the HTTP Routing Mesh, known as HRM, with a new Layer 7 routing and load balancing. This latest enhancement is built upon the new Interlock 2.0 architecture which provides a highly scalable and highly available routing solution for Swarm.  Interlock provides the same functionality as HRM but also includes 2 new features: 1) path based routing and 2) SSL termination.

In the diagram above you can see that a user is browsing to the http://worpress.example.org site.  This request is handled by the Interlock service which then routes the request down to the “wordpress” container.  The website could easily have been secured and utilized path based routing with a URL such as https://example.org/blog.

Stack Deployment

I was excited to try out these features so I chose ehazlett’s demo application to deploy to my lab swarm to see how it works. Here is the contents of the docker stack file. You can see the ehazlett/docker-demo image, the custom ingress overlay network, and three interesting docker labels.

version: '3.4'

services:
  web:
    image: ehazlett/docker-demo:latest
    networks:
      - ingress
    deploy:
      labels:
        com.docker.lb.hosts: interlock-demo.lab.capstonec.net
        com.docker.lb.port: 8080
        com.docker.lb.network: interlock_demo_ingress

networks:
  ingress:
    driver: overlay

The com.docker.lb.hosts and com.docker.lb.port labels simply indicate the host and port that the application will be listening on. The third label com.docker.lb.network indicates the network that interlock proxy will connect to in order to route traffic to my service.

docker stack deploy -c demo-stack.yml interlock_demo

I deploy my application with the command above.  After a moment, I can browse my application as shown below.

The result is the web page above.  All is looking great so far.

Stack Removal Issues

The next thing I wanted to do was to tear it down and move on to something else.  That’s when  problems arose.

docker stack rm interlock_demo
Removing service interlock_demo_web
Removing network interlock_demo_ingress
Failed to remove network pfxwbivrwtpgpqnw828mociy0: Error response from daemon: Error response from daemon: network interlock_demo_ingress id pfxwbivrwtpgpqnw828mociy0 has active endpointsFailed to remove some resources from stack: interlock_demo

So I began to trouble shoot the issue.  If you run the

docker stack ls

command you will see that the interlock_demo stack is gone.  But if you run the

docker network ls

command you will see that the interlock_demo_ingress network still exists.  If you then inspect the network

docker network inspect interlock_demo_ingress
...
        "Containers": {
            "de12019f36cd6c17476182e1e62e2661e35243e24f23254b4f3b4bd00c10a524": {
                "Name": "ucp-interlock-proxy.2.tv8xrwpos77guwoc9praq3l5t",
                "EndpointID": "",
                "MacAddress": "",
                "IPv4Address": "",
                "IPv6Address": ""
            }
        },

you will see that one of the ucp-interlock-proxy services is still attached.  So what are we to do?

Workaround

This is where things get tricky.  Sometimes when I attempt to re-deploy the stack it fails because the network already exists.  And other times the re-deploy succeeds without a problem.

docker stack deploy -c demo-stack.yml interlock_demo
Creating service interlock_demo_web
failed to create service interlock_demo_web: Error response from daemon: network interlock_demo_ingress not found

If you have the former problem its because the ucp-interlock-proxy is still attached to your network. If that is the case then perform the following three steps. This has worked successfully for me.

1) Detach the network from the ucp-interlock-proxy service

docker service update --network-rm interlock_demo_ingress ucp-interlock-proxy

2) Remove the network

docker network rm interlock_demo_ingress

3) Re-deploy your stack

docker stack deploy -c demo-stack.yml interlock_demo

Alternatives

The three step process mentioned previously may work as a manual workaround but it is probably not at the top of my list for a final solution.

One alternative to solving this problem is to create an ingress overlay network outside of your stack, for your application to use…    and just leave it there.

docker network create --driver overlay interlock_ingress

Have your stack attach to it as an external network.  Specify it as the network that the interlock proxy service should use.  Never tear it down.  This way, when you remove your stack, this network is never touched.  Your stack will disconnect from this ingress network while the ucp-interlock-proxy service can stay attached to it forever.

Another alternative is to utilize interlock’s host mode networking. Docker states the following

By default layer 7 routing leverages the Docker Swarm routing mesh, but you don’t have to. You can use host mode networking for maximum performance.

This definitely sounds like something worth looking into but will have to wait for another time.

Conclusion

The Interlock service provides Layer 7 routing for your containerized applications.  It is highly available and automatically configures your applications routing needs.  It is also scalable as you can deploy as many instances of the interlock proxy service as you need.

But what I find most useful is the SSL termination at Interlock as well as path based routing.  These last two features were not available in the previous HRM Layer 7 routing solution.

While we had a few hiccups with how Interlock is working, there are some workarounds and alternative approaches.  With that said, I still think Interlock is a definite improvement to the Docker Enterprise 2.0 platform.

Capstone IT