In this quickstart guide, we will walk you through installing Vector with Helm for a variety of common platforms and tools.
NOTE: The instructions here assume the use of Helm v3, and Bash.
Verify you have helm
and kubectl
installed locally to proceed. helm
should be v3.0+
.
helm version
Ensure you have both the Datadog and Vector charts, and they are up-to-date.
helm repo add datadog https://helm.datadoghq.com helm repo add vector https://helm.vector.dev helm repo update
Access to a pre-existing Kubernetes cluster, or start a new cluster locally with minikube
.
minikube start minikube addons enable metrics-server minikube status
Clone this repository locally to have access to the quickstart values files.
git clone https://github.com/vectordotdev/helm-charts.git && \ cd helm-charts/charts/vector
Export your Datadog API key into your local environment.
export DATADOG_API_KEY="<REPLACE_ME>"
Then install Vector into its own namespace, providing your API key as an environment variable.
helm install vector vector/vector --namespace vector --create-namespace \ --values examples/datadog-values.yaml --set secrets.generic.datadog_api_key="${DATADOG_API_KEY}"
Run the next command to confirm Vector has been configured properly and can connect to your Datadog account.
kubectl logs statefulset/vector --namespace vector --follow
Running the above will tail Vector's logs and allow to confirm your API key is correct. In the logs you should only see INFO messages, and the following logs (one for each configured sink):
{"timestamp":"2022-03-30T20:25:48.016522Z","level":"INFO","message":"Healthcheck: Passed.","target":"vector::topology::builder"} {"timestamp":"2022-03-30T20:25:48.024391Z","level":"INFO","message":"Healthcheck: Passed.","target":"vector::topology::builder"}
Once Vector has been installed and configured, you can install your Datadog Agents. By default, the Agents will forward their logs and metrics directly to Datadog's hosted intake, we'll create a values file to override that behavior to forward to your new Vector Aggregators.
For Datadog Agents version 7.35
/6.35
or greater:
cat <<-'VALUES' > dd-agent.yaml --- datadog: logs: enabled: true containerCollectAll: true agents: useConfigMap: true customAgentConfig: # NOTE: If you're using a quickstart environment like `minikube`, # uncomment the line below; otherwise, we recommend leaving it with the # default setting of `true`. # kubelet_tls_verify: false vector: logs: enabled: true # Use https if SSL is enabled in Vector source configuration url: http://vector-haproxy.vector:8282 metrics: enabled: true # Use https if SSL is enabled in Vector source configuration url: http://vector-haproxy.vector:8282 VALUES
For Datadog Agents version 7.34
/6.34
and older:
NOTE: Metrics will not be routed through Vector for Datadog Agent versions older than 7.35
/6.35
.
cat <<-'VALUES' > dd-agent.yaml --- datadog: logs: enabled: true containerCollectAll: true agents: useConfigMap: true customAgentConfig: # NOTE: If you're using a quickstart environment like `minikube`, # uncomment the line below; otherwise, we recommend leaving it with the # default setting of `true`. # kubelet_tls_verify: false logs_config: logs_dd_url: vector-haproxy.vector:8282 # Set to false if SSL is enabled in Vector source configuration logs_no_ssl: true use_http: true VALUES
NOTE: When you have the datadog_agent
source configured, a helm install
or helm upgrade
will output the appropriate values to provide to the datadog helm chart.
helm install datadog datadog/datadog --namespace datadog --create-namespace \ --values dd-agent.yaml --set datadog.apiKey="${DATADOG_API_KEY}"
Once the Datadog Agents are running you should be able to see your logs in Datadog's Live Tail view, by filtering for sender:vector
. You can also use vector top
to view Vector's topology and related metrics:
kubectl --namespace vector exec --stdin --tty statefulset/vector -- vector top
To diagnose problems the following commands may prove useful:
For Datadog Agents:
kubectl --namespace datadog exec --stdin --tty daemonset/datadog -- agent status
For the HAProxy:
kubectl --namespace vector logs --follow deployment/vector-haproxy
For the Vector Aggregators:
kubectl --namespace vector exec --stdin --tty statefulset/vector -- vector tap internal_logs