# file

The `file` buffer plugin provides a persistent buffer implementation. It uses files to store buffer chunks on disk.

{% hint style="info" %}
In the following examples, it assumes pre-configured chunk keys by default with `<buffer>`. If you want to understand buffer behavior precisely, see [Buffer Section Configuration](https://docs.fluentd.org/configuration/buffer-section) documentation.
{% endhint %}

## Parameters

* [Common Parameters](https://docs.fluentd.org/configuration/plugin-common-parameters)
* [Buffer Section Configurations](https://docs.fluentd.org/configuration/buffer-section)

### `path`

| type   | default | version |
| ------ | ------- | ------- |
| string | nil     | 0.9.0   |

The directory path where buffer chunks are stored. Don't share this directory path with other buffers. Be sure to specify a unique path for each buffer.

```
<match pattern>
  ...
  <buffer>
    @type file
    path /var/log/fluent/buf
  </buffer>
</match>
```

This config outputs the buffer chunk files as follows. The file name is `buffer.b{chunk_id}{path_suffix}`.

```
/var/log/fluentd/buf/buffer.b58eec11d08ca8143b40e4d303510e0bb.log
/var/log/fluentd/buf/buffer.b58eec11d08ca8143b40e4d303510e0bb.log.meta
```

With [multiple workers](https://docs.fluentd.org/deployment/multi-process-workers), a directory is automatically created for each worker. So there is no need to specify a unique path for each worker.

```
<system>
  workers 2
</system>

...

<match pattern>
  ...
  <buffer>
    @type file
    path /var/log/fluent/buf
  </buffer>
</match>
```

This config outputs the buffer chunk files as follows. The directory `worker{worker_id}` is automatically created.

```
/var/log/fluentd/buf/worker0/buffer.b58eec11d08ca8143b40e4d303510e0bb.log
/var/log/fluentd/buf/worker0/buffer.b58eec11d08ca8143b40e4d303510e0bb.log.meta

/var/log/fluentd/buf/worker1/buffer.b5e2a5aca2bcd9818ad6718845ddc456a.log
/var/log/fluentd/buf/worker1/buffer.b5e2a5aca2bcd9818ad6718845ddc456a.log.meta
```

If you specify `root_dir` in [system configuration](https://docs.fluentd.org/deployment/system-config) and [@id](https://docs.fluentd.org/configuration/plugin-common-parameters#id) of the plugin, then you can omit this parameter.

```
<system>
  root_dir /var/log/fluentd
</system>

...

<match pattern>
  @id test_id
  ...
  <buffer>
    @type file
  </buffer>
</match>
```

This config outputs the buffer chunk files as follows. The directory `{root_dir}/worker{worker_id}/{@id}/buffer` is used for the path. In this case, the `worker{worker_id}` directory is created even for a single worker.

```
/var/log/fluentd/worker0/test_id/buffer/buffer.b58eec11d08ca8143b40e4d303510e0bb.log
/var/log/fluentd/worker0/test_id/buffer/buffer.b58eec11d08ca8143b40e4d303510e0bb.log.meta
```

Please make sure that you have **enough space in the path directory**. Running out of disk space is a problem frequently reported by users.

### `path_suffix`

| type   | default | version |
| ------ | ------- | ------- |
| string | .log    | 1.6.3   |

Changes the suffix of the buffer file.

```
# default 
/var/log/fluentd/buf/buffer.b58eec11d08ca8143b40e4d303510e0bb.log
/var/log/fluentd/buf/buffer.b58eec11d08ca8143b40e4d303510e0bb.log.meta

# with 'path_suffix .buf'
/var/log/fluentd/buf/buffer.b58eec11d08ca8143b40e4d303510e0bb.buf
/var/log/fluentd/buf/buffer.b58eec11d08ca8143b40e4d303510e0bb.buf.meta
```

This parameter is useful when `.log` is not fit for your environment. See also [this issue's comment](https://github.com/fluent/fluentd/issues/2236#issuecomment-514733974).

## Tips

### Customize a filename of the buffer chunk

You can customize the prefix of filename (`buffer` by default) by adding `.*` to the end of the `path` parameter.

```
<match pattern>
  ...
  <buffer>
    @type file
    path /var/log/fluent/buf/custom.*
  </buffer>
</match>
```

This config outputs the buffer chunk files as follows. The prefix `buffer` is changed to `custom`.

```
/var/log/fluentd/buf/custom.b58eec11d08ca8143b40e4d303510e0bb.log
/var/log/fluentd/buf/custom.b58eec11d08ca8143b40e4d303510e0bb.log.meta
```

You can also customize the entire filename by adding `.*.` to the `path` parameter.

```
<match pattern>
  ...
  <buffer>
    @type file
    path /var/log/fluent/buf/custom_prefix.*.custom_suffix
  </buffer>
</match>
```

This config outputs the buffer chunk files as follows. In this case, `path_suffix` parameter is not used.

```
/var/log/fluentd/buf/custom_prefix.b58eec11d08ca8143b40e4d303510e0bb.custom_suffix
/var/log/fluentd/buf/custom_prefix.b58eec11d08ca8143b40e4d303510e0bb.custom_suffix.meta
```

## Limitation

{% hint style="warning" %}
Caution: `file` buffer implementation depends on the characteristics of the local file system. Don't use `file` buffer on remote file systems e.g. NFS, GlusterFS, HDFS, etc. We observed major data loss by using the remote file system.
{% endhint %}

{% hint style="warning" %}
`path` is implicitly used as file buffer path if file buffer path is not explicitly configured in `<buffer>` section. In such a case, it must not contain `[` or `]` in the actually evaluated `path`. If such a character was included in the path, it might be trouble (ignored) after restarting of Fluentd during resuming buffer handling process. As a workaround, override `path` parameter in `<buffer>` section which does not contain such a character.
{% endhint %}

If this article is incorrect or outdated, or omits critical information, please [let us know](https://github.com/fluent/fluentd-docs-gitbook/issues?state=open). [Fluentd](http://www.fluentd.org/) is an open-source project under [Cloud Native Computing Foundation (CNCF)](https://cncf.io/). All components are available under the Apache 2 License.