fix: tune net.core.netdev_budget
The default net.core.netdev_budget for the kernel is 300 which is
not enough for most modern systems which have 10G networking, which
is the common case in any OpenStack node. This patch tunes that up
to 1000 which is a much better number for that.
This patch also removes the `NodeSoftNetTimesSqueezed` alert since
it is not actionable and it is something that can normally happen
as a function of a system that has very high traffic. The bigger
concern is the NodeSoftNetDrops which means that there was so much
packets in the queue that were not processed in time that packets
were dropped.
If things take a little longer, there's not that much we can do
about it.
diff --git a/roles/kube_prometheus_stack/files/jsonnet/legacy.libsonnet b/roles/kube_prometheus_stack/files/jsonnet/legacy.libsonnet
index c484a3e..d1fb786 100644
--- a/roles/kube_prometheus_stack/files/jsonnet/legacy.libsonnet
+++ b/roles/kube_prometheus_stack/files/jsonnet/legacy.libsonnet
@@ -165,14 +165,6 @@
name: 'softnet',
rules: [
{
- alert: 'NodeSoftNetTimesSqueezed',
- expr: 'sum(rate(node_softnet_times_squeezed_total[1m])) by (instance) > 10',
- 'for': '10m',
- labels: {
- severity: 'warning',
- },
- },
- {
alert: 'NodeSoftNetDrops',
expr: 'sum(rate(node_softnet_dropped_total[1m])) by (instance) != 0',
'for': '1m',
diff --git a/roles/sysctl/defaults/main.yml b/roles/sysctl/defaults/main.yml
index 787082f..509597e 100644
--- a/roles/sysctl/defaults/main.yml
+++ b/roles/sysctl/defaults/main.yml
@@ -19,6 +19,8 @@
value: 0
- name: net.ipv4.tcp_sack
value: 1
+ - name: net.core.netdev_budget
+ value: 1000
- name: net.core.netdev_max_backlog
value: 250000
- name: net.core.rmem_max