Microservices Service Discovery Design Patterns

This is the 9th post in a series on microservices architecture

When your code invokes a service using a REST API, it needs to know the network location (IP address and port) of the service instance.

The network locations of service instances in a traditional application running on physical hardware are typically static. In modern, cloud-based microservices applications, it’s typically not that straightforward. As a result of autoscaling, failures, and upgrades, service instances change dynamically. You must therefore use service discovery in your client code.

Service Discovery

The IP addresses of the services cannot be configured statically on a client. Instead, dynamic service discovery must be used. Conceptually, service discovery is quite simple. Its main component is a service registry, which contains a list of the network locations of the service instances of an application.

When service instances start and stop, the service registry is updated. The service discovery mechanism queries the service registry to obtain a list of available service instances and routes the request to one of them when a client invokes the service.

Application-level service discovery patterns

The application’s services and its clients can interact with the service registry to implement service discovery. Each service instance registers its network location with the service registry. A client requests a list of service instances from the service registry before invoking a service. The client then sends a request to one of those instances.

This approach is a combination of two patterns.

1. Self-registration pattern — During startup, a service instance invokes the registry’s registration API to register its network location. A service registry may require that the service instance invoke a heartbeat API periodically to prevent its registration from expiring. When the service instance shuts down, it unregisters itself from the service registry.
2. Client-side discovery pattern — To invoke a service, a service client queries the service registry for a list of the service’s instances. A load-balancing algorithm is used by the service client to select an instance of the service. The client then requests the selected instance.

Netflix and Pivotal have popularized application-level service discovery. For example, Netflix’s Eureka is a highly available service registry. Netflix components can be easily used with Spring Cloud, a Spring-based framework developed by Pivotal. Spring Cloud-based services register with Eureka, and Spring Cloud-based clients use Eureka to discover services.

Drawbacks of Application-level service discovery

• Language-specific service discovery libraries are required.
• It is your responsibility to set up and manage the service registry.
• When a service instance is running but not handling requests, it often lacks the self-awareness to unregister from the service registry.

Platform-provided service discovery patterns

Many modern deployment platforms, such as Docker and Kubernetes, have built-in service registries and service discovery mechanisms. Each service is assigned a DNS name, a virtual IP (VIP) address, and a DNS name that resolves to the VIP address.

A service client requests the DNS name/VIP, and the deployment platform automatically routes the request to one of the available service instances. In this way, service registration, service discovery, and request routing are entirely handled by the deployment platform. A service registry tracks the IP addresses of deployed services in the deployment platform.

This approach is a combination of two patterns:

1. 3rd Party registration pattern —Rather than a service registers itself with the service registry, a third party called the registrar, which is typically part of the deployment platform, does it instead. The registrar registers the service instance with the service registry when it starts up. The registrar unregisters the service instance from the service registry when the instance shuts down.
2. Server-side discovery pattern — Instead of a client querying the service registry, it makes a request to a DNS name, which resolves to a request router that queries the registry and loads balances requests. The AWS Elastic Load Balancer (ELB) is an example of a server-side discovery router. A client sends HTTP/TCP requests to the ELB, which load balances traffic among a set of EC2 instances. An ELB also acts as a service registry. Instances are registered with the ELB either explicitly via an API call or automatically as part of an auto-scaling group.

Benefits of Platform-based service discovery

• The deployment platform handles all aspects of service discovery.
• Neither the services nor the clients contain any code for service discovery.
• Service discovery is available to all services and clients, regardless of the language in which they are written.

Drawbacks of Platform-based service discovery

• Only services that have been deployed using the platform can be discovered.