TDengine as a TiDB Alternative Fuels Connected Vehicle Technology for Li Auto

Who is Li Auto?

Li Auto Inc. (NASDAQ: LI) is an innovator in China’s new energy vehicle market. The Company designs, develops, manufactures, and sells premium smart electric vehicles. Through innovations in product, technology, and business model, the Company provides families with safe, convenient, and refined products and services. Li Auto is a pioneer to successfully commercialize extended-range electric vehicles in China. Its first model, Li ONE, is a six-seat, large premium electric SUV equipped with a range extension system and advanced smart vehicle solutions. The Company started volume production of Li ONE in November 2019 and released the 2021 Li ONE in May 2021. The Company leverages technology to create value for its users. It concentrates its in-house development efforts on its proprietary range extension system, next-generation electric vehicle technology, and smart vehicle solutions. Beyond Li ONE, the Company will expand its product line by developing new vehicles, including BEVs and EREVs, to target a broader consumer base.

Usage Scenario & Bottlenecks

Data transmission is a necessity in the connected car scenario. We need to write timestamped data that is tagged with device information to DBMS via a cloud platform. Currently, the average write QPS is 70,000, while it might exceed 200,000 write QPS in the coming three years. We once utilized MangoDB to store metrics, but MangoDB showed various deficiencies in the actual use. Afterward, considering the requirement of horizontal scaling, we transferred data to TiDB. However, TiDB failed to fit in our usage scenario as its writing performance didn’t satisfy our expectations. Besides, high-concurrency writing cannot be well supported by Hybrid Transactional/Analytical Processing (HTAP) database, rendering high storage costs.

From our perspective, TiDB is more suitable for transactional/analytical processing scenarios, moreover, it has a high requirement of hardware configuration. Thus, TiDB is not cost-effective for time-series data processing. Additionally, we had to create databases and tables by the order of months. Furthermore, we also needed to define the tag for each data collection. Given the experience of using TiDB, we summarized bottlenecks and requirements below:

  • Constantly high-concurrency writing with tags leads to disordered timestamps.
  • Considering the rapid growth of data volumes, transparent scaling out is required.
  • Aggregations over intervals are needed.
  • Due to large data volumes, industry-leading storage efficiency is required for lower storage costs.
  • Creating databases & tables by the order of months causes higher O&M costs.
  • TTL design is necessary.
  • One data collection point for one table is not supported.
  • Writing data in batches is not supported.
  • Latency for data writing is higher than the expectation.

Considering the bottlenecks and requirements mentioned above, we decided to try the solution provided by TDengine, a carrier-grade time-series database (TSDB) specialized for time-series data processing. According to the results of performance testing, TDengine demonstrated higher performance than our expectation.

The powerful features of TDengine shown below perfectly fit our usage scenario:

  • Row-based storage in memory & column-based storage in the hard disk, rendering faster insertion speed and lower computing resource consumption
  • Excellent time-series data compression efficiency
  • One data point (device) for one table
  • Multi-thread batch writing
  • Transparent scaling-out & scaling-in
  • TTL support

In light of its excellent high-concurrency writing performance and impressive data compression efficiency, we decided to migrate data from TiDB to TDengine.

Data Migration and Use Cost Comparison

Migration Procedures

It took three steps to migrate data to TDengine:

  1. First, writing incremental data to TDengine and storing historical data to TiDB
  2. Second, formatting and migrating historical data to TDengine
  3. Deployment pathway: Domain Name — → LB — → FirstEP+SecondEP

TiDB VS TDengine

Conclusion

The competitive advantages of TDengine:

  • A carrier-grade TSDB outperformed InfluxDB in terms of write/read speed and storage efficiency
  • Row-based storage in MEM and column-based storage in the disk
  • STable tailored for the typical characteristics of IoT big data processing scenarios
  • Much lower O&M costs
  • Flexible scaling-out design
  • Shorter response time (RT) for aggregated queries
  • TTL support

Suggestions for TDengine:

  • Efficient troubleshooting tool and versatile data visualization design
  • Lacking a fully-featured tool kit tailored for TDengine
  • The strong consistency model of TDengine requires all servers and clients to update in a synchronous manner, rendering relatively more time and resource consumption.
  • Error reporting function should be more user-friendly for troubleshooting.
  • “prepare statement” is not supported by the SDK based on Go (It has been implemented in the latest version).
  • Account isolation cannot be supported in the same cluster.

No matter it is MySQL, MangoDB, TiDB, or TDengine, there is no so-called perfect DBMS. Different usage scenarios have varied requirements, hence each DBMS is an excellent DBMS as long as it is tailored for certain usage scenarios.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
TDengine

TDengine

Open-source, cloud-native time-series database optimized for IoT. See more at https://tdengine.com or https://github.com/taosdata/TDengine