# Kubernetes ![Fluentd + Kubernetes](/files/uqyvmpjHxfTAKGB3kozD) [Kubernetes](http://kubernetes.io) provides two logging endpoints for applications and cluster logs: 1. Stackdriver Logging for use with Google Cloud Platform; and, 2. Elasticsearch. Behind the scenes, there is a logging agent that takes care of the log collection, parsing and distribution: [Fluentd](http://www.fluentd.org). This document focuses on how to deploy Fluentd in Kubernetes and extend the possibilities to have different destinations for your logs. ## Getting Started This document assumes that you have a Kubernetes cluster running or at least a local (single) node that can be used for testing purposes. Before getting started, make sure you understand or have a basic idea about the following concepts from Kubernetes: * [Node](https://kubernetes.io/docs/concepts/architecture/nodes/) > A node is a worker machine in Kubernetes, previously known as a minion. A node may be a VM or physical machine, depending on the cluster. Each node has the services necessary to run pods and is managed by the master components... * [Pod](https://kubernetes.io/docs/concepts/workloads/pods/) > A pod (as in a pod of whales or pea pod) is a group of one or more containers (such as Docker containers), the shared storage for those containers, and options about how to run the containers. Pods are always co-located and co-scheduled, and run in a shared context... * [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) > A DaemonSet ensures that all (or some) nodes run a copy of a pod. As nodes are added to the cluster, pods are added to them. As nodes are removed from the cluster, those pods are garbage collected. Deleting a DaemonSet will clean up the pods it created... Since applications runs in Pods and multiple Pods might exists across multiple nodes, we need a specific Fluentd-Pod that takes care of log collection on each node: Fluentd DaemonSet. ## Fluentd DaemonSet For [Kubernetes](https://kubernetes.io), a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) ensures that all (or some) nodes run a copy of a *pod*. To solve log collection, we are going to implement a Fluentd DaemonSet. Fluentd is flexible enough and has the proper plugins to distribute logs to different third-party applications like databases or cloud services, so the principal question is *Where will the logs be stored?*. Once we got that question answered, we can move forward configuring our DaemonSet. The following steps will focus on sending the logs to an Elasticsearch Pod: ### Get Fluentd DaemonSet sources We have created a Fluentd DaemonSet that have the proper rules and container image ready to get started: * Please grab a copy of the repository from the command line using GIT: ``` $ git clone https://github.com/fluent/fluentd-kubernetes-daemonset ``` ### DaemonSet Content The cloned repository contains several configurations that allow to deploy Fluentd as a DaemonSet. The Docker container image distributed on the repository also comes pre-configured so that Fluentd can gather all the logs from the Kubernetes node's environment and append the proper metadata to the logs. This repository has several presets for Alpine/Debian with popular outputs: * [DaemonSet preset settings](https://github.com/fluent/fluentd-kubernetes-daemonset/tree/master/docker-image/v1.14) ## Logging to Elasticsearch ### Requirements From the `fluentd-kubernetes-daemonset/` directory, find the YAML configuration file: * [`fluentd-daemonset-elasticsearch.yaml`](https://github.com/fluent/fluentd-kubernetes-daemonset/blob/master/fluentd-daemonset-elasticsearch.yaml) As an example, let's see a part of this file: ``` apiVersion: apps/v1 kind: DaemonSet metadata: name: fluentd namespace: kube-system ... spec: ... spec: containers: - name: fluentd image: quay.io/fluent/fluentd-kubernetes-daemonset env: - name: FLUENT_ELASTICSEARCH_HOST value: "elasticsearch-logging" - name: FLUENT_ELASTICSEARCH_PORT value: "9200" - name: FLUENT_ELASTICSEARCH_SSL_VERIFY value: "true" - name: FLUENT_ELASTICSEARCH_SSL_VERSION value: "TLSv1_2" ... ``` This YAML file contains two relevant environment variables that are used by Fluentd when the container starts: | Environment Variable | Description | Default | | ---------------------------------- | --------------------------------------- | :---------------------: | | `FLUENT_ELASTICSEARCH_HOST` | Specify the host name or IP address. | `elasticsearch-logging` | | `FLUENT_ELASTICSEARCH_PORT` | Elasticsearch TCP port | 9200 | | `FLUENT_ELASTICSEARCH_SSL_VERIFY` | Whether verify SSL certificates or not. | `true` | | `FLUENT_ELASTICSEARCH_SSL_VERSION` | Specify the version of TLS. | `TLSv1_2` | Any relevant change needs to be done in the YAML file before deployment. The defaults assume that at least one Elasticsearch Pod **elasticsearch-logging** exists in the cluster. If this article is incorrect or outdated, or omits critical information, please [let us know](https://github.com/fluent/fluentd-docs-gitbook/issues?state=open). [Fluentd](http://www.fluentd.org/) is an open-source project under [Cloud Native Computing Foundation (CNCF)](https://cncf.io/). All components are available under the Apache 2 License. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.fluentd.org/container-deployment/kubernetes.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.