Intro: For those who are familiar with TDengine should be aware that TDengine has outstanding performance when processing time-series data. This is based on the insight and summarization of the characteristics of time-series data. By utilizing the characteristics of time-series data, TDengine designed the storage engine and the querying engine. This article primarily introduces the innovative storage engine of TDengine and examines why it helps to reach high performance.
01 Characteristics of Time-Series Data
Let’s observe some time-series data, and try to summarize characteristics of time-series data.
The figure above shows some classic time-series data. The horizontal axis represents time, and the vertical axis represents the collected data at each timestamp. The figure tells us a lot:
- Some time-series data keep constant for a long time with anomalous fluctuations at some point
- Some time-series data changing in a trend in a period
- Some time-series data fluctuate around a constant value in a certain range with different frequencies
Taking advantage of these characteristics, we designed a specific storage engine for time-series data processing.
02 The Data Model of TDengine
TDengine’s data model mainly has the following characteristics:
- One table for one data collection point
- Data in a table is contiguously stored in the form of block in a file
- The size of a data block can be configured
- Use Block Range INdex (BRIN) technique
One table for one data collection point will end with a lot of tables. In order to solve the problem, we introduce vnode as the sharding unit. Click “The Detailed Explanation of the Design Thinking of TDengine 2.0’s Overall Structure” to learn about the details of the concept of vnode.
In summary, vnode is the basic unit of time-series data storage, and a vnode contains all the data of partial tables. The load balance and data replication both are processed on vnode, which can take advantage of multi-core to improve performance.
In a vnode, data partitioned in multiple file groups based on data time range. Consisted with multiple files, a file group stores data in a certain time range. Given the data timestamp, we can efficiently find which file the data is in and vice versa.
03 TSDB Storage Engine
TSDB is TDengine’s storage engine developed to efficiently process time-series data by taking advantage of the data features. In addition to storage interfaces, TSDB also provides query interfaces. TSDB stores the META data and time-series data for the tables in a vnode, where time-series data can be stored in both row and columnar format (Row storage is available after TDengine 2.0). In RAM, time-series data is indexed through SkipList, whereas in hard disk, it is indexed through Block Range INdex (BRIN).
TSDB stores the META data for tables in a vnode. META includes SCHEMA of Tables/Super Tables, child tables’ TAG values, TAG SCHEMA and child tables / Super Tables‘ dependability. The creation, update, and deletion and other executions of META data will be performed in RAM first, and then serialized and written into hard disk.
In TSDB, META data is fully cached, based on the first TAG value of the child table, a memory query is built. Because query is performed to the first value of TAG, the speed is the fastest. Using vnode’s Sharding technique can fully utilize many vnode resources to perform the filter query execution of the tables.
META Data Persistent Storage
When META data is written into RAM, serialized record will be created at the same time, and stored into RAM’s buffer using the append only form. As RAM data reaches a certain amount, commit operation will be triggered. During the commit, updated serialized META data will be written into the META file in the hard disk in the append only form. Every table’s latest status, the update and deletion of tables will be appended to the META file, and serialize into a record.
TSDB is also responsible for storing the time-series data (collected data) for tables in a vnode. Time-series data will be written into TSDB’s pre-allocated RAM buffer region, and when the data in the buffer region reaches a certain amount, commit will be triggered, and persistent storage will be performed.
The time-series data in TSDB’s memory uses row storage, which can support adding buffer using the append only technique, so that allocated storage resources can be fully utilized to buffer enough data to perform commit, that is an accumulation beneficial to data of a block amount to compress. To make querying and processing random data easier, a SkipList is built in the memory as a memory query. Memory also maintains written data’s latest and oldest time and other informations. A data’s row storage format in memory is as shown below.
Time-Series Data Persistent Storage
When data within TSDB’s memory reaches a certain amount, commit will be triggered. During the commit, time-series data changes from row storage format to column storage format. In addition, BRIN query is maintained to introduce the LAST file and SUB-BLOCK mechanism to process file fragmentation. Row storage format is as shown below.
When TSDB is started, a BUFFER POOL is pre-allocated as the writing buffer (16 MB*6=96 MB by default), buffer block’s size and number can be paired, and the block’s number can be edited. META data and time-series data applies for writing space from the buffer block, the writing engine applies for buffer clock from BUFFER POOL, and commit is triggered when fully written buffer block occupies 1/3 of the total buffer block. During the commit, the buffer block’s data is written into META and other files, and buffer blocks are return to the BUFFER POOL after the placement is ends, forming a cycling mechanism. Merged query is performed during query for MEM, IMEM, and data in data files as shown in the figure below.
Advantages of TSDB’s Design
- High efficiency for query in a time range for a single table
- Memory is fully utilized and can buffer more data
- The columnar format for data in file can achieve higher data compression ratio
- Avoid extra file merge compared with LSM
- Tag data is stored separately from the time-series data
This article primarily explained the TDengine’s storage engine. However, fine-tuned compression algorithms and the design of the querying model also leads to TDengine’s high performance and low storage space. Follow us for upcoming contents to explain why TDengine has such high performance.
About The Author: Hongze Cheng, co-founder of Tao’s Data, key contributor of TDengine. Hongze received his Master’s degree in Electrical Engineering from University of Michigan, and his Bachelor’s degree from University of Science and Technology of China, where he received the Guo Moruo Scholarship. Hongze’s research topics are primarily focus on the IoT data and machine learning.