geoip

Search…
geoip
The out_geoip Buffered Output plugin adds geographic location information to logs using the Maxmind GeoIP databases. This document doesn't describe all parameters. If you want to know full features, check the Further Reading section.
Prerequisites
The GeoIP library.
:::term # for RHEL/CentOS $ sudo yum install geoip-devel --enablerepo=epel
# for Ubuntu/Debian $ sudo apt-get install libgeoip-dev
# for MacOSX (brew) $ brew install geoip
Install
out_geoip is not included in td-agent. All users must install the fluent-plugin-geoip gem using the following command.
$ fluent-gem install fluent-plugin-geoip
$ sudo /usr/sbin/td-agent-gem install fluent-plugin-geoip
Example Configuration
The configuration shown below adds geolocation information to apache.access
<match test.message>
  @type geoip
  geoip_lookup_key        host
  enable_key_country_code geoip_country
  enable_key_city         geoip_city
  enable_key_latitude     geoip_lat
  enable_key_longitude    geoip_lon
  remove_tag_prefix       test.
  add_tag_prefix          geoip.
  flush_interval          5s
</match>
​
​
:::text
# original record
test.message {
  "host":"66.102.9.80",
  "message":"test"
}
​
# output record
geoip.message: {
  "host":"66.102.9.80",
  "message":"test",
  "geoip_country":"US",
  "geoip_city":"Mountain View",
  "geoip_lat":37.4192008972168,
  "geoip_lon":-122.05740356445312
}
Please see the fluent-plugin-geoip README for further details.
Parameters
geoip_lookup_key (required)
Specifies the geoip lookup field (default: host) If accessing a nested hash value, delimit the key with '.', as in 'host.ip'.
remove_tag_prefix / add_tag_prefix (requires one or the other)
Set tag replace rule.
enable_key_*** (requires at least one)
Specifies the geographic data that will be added to the record. The supported parameters are shown below:
enable_key_city
enable_key_latitude
enable_key_longitude
enable_key_country_code3
enable_key_country_code
enable_key_country_name
enable_key_dma_code
enable_key_area_code
enable_key_region
include_tag_key
Set to true to include the original tag name in the record. (default: false)
tag_key
Adds the tag name into the record using this value as the key name When include_tag_key is set to true.
Buffer Parameters
For advanced usage, you can tune Fluentd's internal buffering mechanism with these parameters.
buffer_type
The buffer type is memory by default (buf_memory). The file (buf_file) buffer type can be chosen as well. Unlike many other output plugins, the buffer_path parameter MUST be specified when using buffer_type file.
buffer_queue_limit, buffer_chunk_limit
The length of the chunk queue and the size of each chunk, respectively. Please see the Buffer Plugin Overview article for the basic buffer structure. The default values are 64 and 256m, respectively. The suffixes "k" (KB), "m" (MB), and "g" (GB) can be used for buffer_chunk_limit.
flush_interval
The interval between forced data flushes. The default is nil (don't force flush and wait until the end of time slice + time_slice_wait). The suffixes "s" (seconds), "m" (minutes), and "h" (hours) can be used.
log_level option
The log_level option allows the user to set different levels of logging for each plugin. The supported log levels are: fatal, error, warn, info, debug, and trace.
Please see the logging article for further details.
Use Cases
Plot real time access statistics on a world map using Elasticsearch and Kibana
The country_code field is needed to visualize access statistics on a world map using Kibana.
Note: The following plugins are required: * fluent-plugin-geoip * fluent-plugin-elasticsearch
<match td.apache.access>
  @type geoip
​
  # Set key name for the client ip address values
  geoip_lookup_key     host
​
  # Specify key name for the country_code values
  enable_key_country_code  geoip_country
​
  # Swap tag prefix from 'td.' to 'es.'
  remove_tag_prefix    td.
  add_tag_prefix       es.
</match>
​
<match es.apache.access>
  @type            elasticsearch
  host            localhost
  port            9200
  type_name       apache
  logstash_format true
  flush_interval  10s
</match>
Further Reading
​fluent-plugin-geoip repository​
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
copy
roundrobin
Last modified 2yr ago
Copy link
Outline
Prerequisites
Install
Example Configuration
Parameters
geoip_lookup_key (required)
remove_tag_prefix / add_tag_prefix (requires one or the other)
enable_key_*** (requires at least one)
buffer_queue_limit, buffer_chunk_limit
flush_interval
Use Cases