running a victoriametrics cluster
I recently had a need to work with metrics, and looking at the landscape of modern tools, went with victoriametrics. After initially using the single binary version, I went on to setup the cluster version, using two nodes for everything (free nodes in oracles free tier!) run these binaries on each node (where 10.0.2.41 and 10.0.2.40 are the addresses of the nodes)
./vmstorage-prod -retentionPeriod 5y -storageDataPath /var/lib/victoriametrics
./vminsert-prod -storageNode=10.0.2.41:8400 -storageNode=10.0.2.40:8400 -replicationFactor=2
./vmselect-prod -storageNode=10.0.2.41:8401 -storageNode=10.0.2.40:8401 -replicationFactor=2 -dedup.minScrapeInterval=1ms
front vmselect and vminsert with nginx
server {
listen 443 ssl;
server_name metrics.foo.bar;
location /insert/ {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://localhost:8480;
}
location /select/ {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://localhost:8481;
}
use certbot to get a letsencrypt cert, it will configure nginx for you (if you installed python3-certbot-nginx ) There is no auth of any kind, because i chose to front it with cloudflare, using the loadbalancer function to front the two nodes , and locked down nginx to cloudflare ips
server {
include /etc/nginx/cloudflare-allow.conf;
allow 127.0.0.1/32;
deny all;
in cloudflare I setup that a cert is required to access the fqdn , therefore making the whole system require cert auth to read or write metrics I send metrics with telegraf , and this config
[[outputs.influxdb]]
urls = ["https://metrics.foo.bar:443/insert/0/influx"]
database = "whatever"
tls_cert = "/etc/telegraf/cloudflare.user.crt"
tls_key = "/etc/telegraf/cloudflare.user_nopass.key"
and grafana works fine with those certs too
Comments