Customize CoreDNS with Azure Kubernetes Service
Azure Kubernetes Service (AKS) uses the CoreDNS project for cluster DNS management and resolution with all 1.12.x and higher clusters. For more information about CoreDNS customization and Kubernetes, see the official upstream documentation.
AKS is a managed service, so you can't modify the main configuration for CoreDNS (a CoreFile). Instead, you use a Kubernetes ConfigMap to override the default settings. To see the default AKS CoreDNS ConfigMaps, use the kubectl get configmaps --namespace=kube-system coredns -o yaml command.
This article shows you how to use ConfigMaps for basic CoreDNS customization options of in AKS. This approach differs from configuring CoreDNS in other contexts, such as CoreFile.
Note
Previously, kube-dns was used for cluster DNS management and resolution, but it's now deprecated. kube-dns offered different customization options via a Kubernetes config map. CoreDNS is not backwards compatible with kube-dns. Any customizations you previously used must be updated for CoreDNS.
Before you begin
- This article assumes that you have an existing AKS cluster. If you need an AKS cluster, you can create one using Azure CLI, Azure PowerShell, or the Azure portal.
- Verify the version of CoreDNS you're running. The configuration values may change between versions.
- When you create configurations like the examples below, your names in the data section must end in .server or .override. This naming convention is defined in the default AKS CoreDNS ConfigMap, which you can view using the
kubectl get configmaps --namespace=kube-system coredns -o yamlcommand.
Plugin support
All built-in CoreDNS plugins are supported. No add-on/third party plugins are supported.
Rewrite DNS
You can customize CoreDNS with AKS to perform on-the-fly DNS name rewrites.
Create a file named
corednsms.yamland paste the following example configuration. Make sure to replace<domain to be rewritten>with your own fully qualified domain name.apiVersion: v1 kind: ConfigMap metadata: name: coredns-custom namespace: kube-system data: test.server: | <domain to be rewritten>.com:53 { log errors rewrite stop { name regex (.*)\.<domain to be rewritten>.com {1}.default.svc.cluster.local answer name (.*)\.default\.svc\.cluster\.local {1}.<domain to be rewritten>.com } forward . /etc/resolv.conf # you can redirect this to a specific DNS server such as 10.0.0.10, but that server must be able to resolve the rewritten domain name }Important
If you redirect to a DNS server, such as the CoreDNS service IP, that DNS server must be able to resolve the rewritten domain name.
Create the ConfigMap using the
kubectl apply configmapcommand and specify the name of your YAML manifest.kubectl apply -f corednsms.yamlVerify the customizations have been applied using the
kubectl get configmapsand specify your coredns-custom ConfigMap.kubectl get configmaps --namespace=kube-system coredns-custom -o yamlForce CoreDNS to reload the ConfigMap using the
kubectl delete podcommand and thekube-dnslabel. This command deletes thekube-dnspods, and then the Kubernetes Scheduler recreates them. The new pods contain the change in TTL value.kubectl delete pod --namespace kube-system -l k8s-app=kube-dns
Custom forward server
If you need to specify a forward server for your network traffic, you can create a ConfigMap to customize DNS.
Create a file named
corednsms.yamland paste the following example configuration. Make sure to replace theforwardname and the address with the values for your own environment.apiVersion: v1 kind: ConfigMap metadata: name: coredns-custom namespace: kube-system data: test.server: | # you may select any name here, but it must end with the .server file extension <domain to be rewritten>.com:53 { forward foo.com 1.1.1.1 }Create the ConfigMap using the
kubectl apply configmapcommand and specify the name of your YAML manifest.kubectl apply -f corednsms.yamlForce CoreDNS to reload the ConfigMap using the
kubectl delete podso the Kubernetes Scheduler can recreate them.kubectl delete pod --namespace kube-system -l k8s-app=kube-dns
Use custom domains
You may want to configure custom domains that can only be resolved internally. For example, you may want to resolve the custom domain puglife.local, which isn't a valid top-level domain. Without a custom domain ConfigMap, the AKS cluster can't resolve the address.
Create a new file named
corednsms.yamland paste the following example configuration. Make sure to update the custom domain and IP address with the values for your own environment.apiVersion: v1 kind: ConfigMap metadata: name: coredns-custom namespace: kube-system data: puglife.server: | # you may select any name here, but it must end with the .server file extension puglife.local:53 { errors cache 30 forward . 192.11.0.1 # this is my test/dev DNS server }Create the ConfigMap using the
kubectl apply configmapcommand and specify the name of your YAML manifest.kubectl apply -f corednsms.yamlForce CoreDNS to reload the ConfigMap using the
kubectl delete podso the Kubernetes Scheduler can recreate them.kubectl delete pod --namespace kube-system -l k8s-app=kube-dns
Stub domains
CoreDNS can also be used to configure stub domains.
Create a file named
corednsms.yamland paste the following example configuration. Make sure to update the custom domains and IP addresses with the values for your own environment.apiVersion: v1 kind: ConfigMap metadata: name: coredns-custom namespace: kube-system data: test.server: | # you may select any name here, but it must end with the .server file extension abc.com:53 { errors cache 30 forward . 1.2.3.4 } my.cluster.local:53 { errors cache 30 forward . 2.3.4.5 }Create the ConfigMap using the
kubectl apply configmapcommand and specify the name of your YAML manifest.kubectl apply -f corednsms.yamlForce CoreDNS to reload the ConfigMap using the
kubectl delete podso the Kubernetes Scheduler can recreate them.kubectl delete pod --namespace kube-system -l k8s-app=kube-dns
Hosts plugin
All built-in plugins are supported, so the CoreDNS hosts plugin is available to customize as well.
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns-custom # this is the name of the configmap you can overwrite with your changes
namespace: kube-system
data:
test.override: | # you may select any name here, but it must end with the .override file extension
hosts {
10.0.0.1 example1.org
10.0.0.2 example2.org
10.0.0.3 example3.org
fallthrough
}
Troubleshooting
For general CoreDNS troubleshooting steps, such as checking the endpoints or resolution, see Debugging DNS resolution.
Configure CoreDNS pod scaling
Sudden spikes in DNS traffic within AKS clusters are a common occurrence due to the elasticity that AKS provides for workloads. These spikes can lead to an increase in memory consumption by CoreDNS pods. In some cases, this increased memory consumption could cause Out of memory issues. To preempt this issue, AKS clusters auto scale CoreDNS pods to reduce memory usage per pod. The default settings for this auto scaling logic are stored in the coredns-autoscaler ConfigMap. However, you may observe that the default auto scaling of CoreDNS pods is not always aggressive enough to prevent Out of memory issues for your CoreDNS pods. In this case, you can directly modify the coredns-autoscaler ConfigMap. Please note that simply increasing the number of CoreDNS pods without addressing the root cause of the Out of memory issue may only provide a temporary fix. If there is not enough memory available across the nodes where the CoreDNS pods are running, increasing the number of CoreDNS pods will not help. You may need to investigate further and implement appropriate solutions such as optimizing resource usage, adjusting resource requests and limits, or adding more memory to the nodes.
CoreDNS uses horizontal cluster proportional autoscaler for pod auto scaling. The coredns-autoscaler ConfigMap can be edited to configure the scaling logic for the number of CoreDNS pods. The coredns-autoscaler ConfigMap currently supports two different ConfigMap key values: linear and ladder which correspond to two supported control modes. The linear controller yields a number of replicas in [min,max] range equivalent to max( ceil( cores * 1/coresPerReplica ) , ceil( nodes * 1/nodesPerReplica ) ). The ladder controller calculates the number of replicas by consulting two different step functions, one for core scaling and another for node scaling, yielding the max of the two replica values. For more information on the control modes and ConfigMap format, please consult the upstream documentation.
To retrieve the coredns-autoscaler ConfigMap, you can run the kubectl get configmap coredns-autoscaler -n kube-system -o yaml command which will return the following:
apiVersion: v1
data:
ladder: '{"coresToReplicas":[[1,2],[512,3],[1024,4],[2048,5]],"nodesToReplicas":[[1,2],[8,3],[16,4],[32,5]]}'
kind: ConfigMap
metadata:
name: coredns-autoscaler
namespace: kube-system
resourceVersion: "..."
creationTimestamp: "..."
Enable DNS query logging
Add the following configuration to your coredns-custom ConfigMap:
apiVersion: v1 kind: ConfigMap metadata: name: coredns-custom namespace: kube-system data: log.override: | # you may select any name here, but it must end with the .override file extension logApply the configuration changes and force CoreDNS to reload the ConfigMap using the following commands:
# Apply configuration changes kubectl apply -f corednsms.yaml # Force CoreDNS to reload the ConfigMap kubectl delete pod --namespace kube-system -l k8s-app=kube-dnsView the CoreDNS debug logging using the
kubectl logscommand.kubectl logs --namespace kube-system -l k8s-app=kube-dns
Next steps
This article showed some example scenarios for CoreDNS customization. For information on the CoreDNS project, see the CoreDNS upstream project page.
To learn more about core network concepts, see Network concepts for applications in AKS.
Feedback
Submit and view feedback for