Building a High-Performance Time Series Data Warehouse for IoT Dashboards with VictoriaMetrics on VPS
Transform IoT sensor data into valuable insights by deploying a high-performance time series data warehouse on a VPS using VictoriaMetrics. This guide provides a comprehensive...
Image credit: herbert2512 · Pixabay Content License
Transform IoT sensor data into valuable insights by deploying a high-performance time series data warehouse on a VPS using VictoriaMetrics. This guide provides a comprehensive architecture, optimized configuration, and execution commands to deploy a real-time data store capable of handling millions of data points per second, ensuring scalability and low operational costs. Leveraging NVMe Cloud VPS, the system can manage high loads without physical infrastructure investment, ideal for small to medium-scale IoT projects.
Overview of Time Series Systems for IoT
In IoT systems, sensor data is continuously collected over time, forming time series data streams. Systems such as Prometheus, InfluxDB, and VictoriaMetrics are specifically designed to efficiently store and query this data. VictoriaMetrics stands out for its high performance, low cost, and excellent scalability, making it ideal for VPS environments with limited resources. With its simple architecture and superior performance, VictoriaMetrics is the ideal choice for building a real-time time series data warehouse for IoT dashboards.
Deploying on a VPS (NVMe Cloud VPS) offers flexibility and lower operational costs compared to physical servers. Modern VPS platforms provide NVMe SSDs, enabling fast read/write operations essential for time series applications processing large volumes of data. Additionally, virtualization-based deployment allows easy backup, recovery, and resource scaling when needed.
Preparation and Installing VictoriaMetrics
To begin, you need a Linux-based VPS (Ubuntu 20.04/22.04 is recommended). Access the VPS via SSH and update the system:
sudo apt update && sudo apt upgrade -y
Next, install VictoriaMetrics by downloading the package from the official site:
curl -s https://api.victoriametrics.com/download/victoria-metrics-pro | tar -xzf - -C /usr/local/bin --strip-components=1
sudo chmod +x /usr/local/bin/victoria-metrics-pro
To manage the service, create a systemd file:
sudo nano /etc/systemd/system/victoria-metrics.service
Insert the following content:
[Unit]
Description=VictoriaMetrics
After=network.target
[Service]
Type=simple
User=root
ExecStart=/usr/local/bin/victoria-metrics-pro --storageDataPath=/var/lib/victoria-metrics --httpListenAddr=:8428
Restart=always
[Install]
WantedBy=multi-user.target
Start the service:
sudo systemctl daemon-reload
sudo systemctl start victoria-metrics
sudo systemctl enable victoria-metrics
Integration with IoT and Dashboards
To send data from IoT sensors to VictoriaMetrics, you can use clients such as MQTT, HTTP API, or SDKs for Python, Node.js, etc. Below is an example of sending data via HTTP:
curl -X POST http://:8428/api/v1/write \
-H "Content-Type: application/json" \
-d '{
"metric": "sensor_temperature",
"timestamp": 1700000000,
"value": 25.3,
"tags": {
"device": "sensor-01",
"location": "floor-1"
}
}'
To visualize the data, integrate with a dashboard like Grafana. Install Grafana on the same VPS or a separate server:
sudo apt install -y grafana
sudo systemctl start grafana-server
sudo systemctl enable grafana-server
In Grafana, add a data source with URL: http://
Performance Tuning and Optimization
To optimize performance, adjust the following parameters in the VictoriaMetrics configuration file:
- storageDataPath: Set to an NVMe drive or a large-capacity directory.
- maxConcurrentInserts: Increase to 100–200 if many devices are sending data.
- memoryLimit: Set an appropriate memory limit based on VPS resources.
Monitor performance using tools like Prometheus and Grafana, tracking metrics such as CPU, RAM, I/O, and request processing time.
Testing and Troubleshooting
To verify that VictoriaMetrics is running, access the following URL:
http://:8428/api/v1/status
If no response is received, check the logs:
sudo journalctl -u victoria-metrics -n 50 --no-pager
If data does not appear on the dashboard, verify network connectivity, API configuration, and data format. Ensure that tags and metrics are correctly defined.
Reference Resources
For more information on VictoriaMetrics, consult the official documentation: https://victoriametrics.com/docs/. For guidance on high-performance VPS configuration, refer to: /en/vps.
Related infrastructure & engineering guides: Optimizing PocketBase on VPS to Achieve 30,000 Requests/Second for Single File Backend Systems
Recommended NVMe Cloud VPS Plans to Deploy This Setup
Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.
VPS-3E
10% VAT included
- CPU: 2 vCPU
- RAM: 4 GB
- SSD: 50 GB NVMe
