Modern enterprises collect and analyze vast amounts of data for a variety of use cases. Sumo Logic customers use ingested data to monitor operations, troubleshoot problems, to understand and better serve customers, to ensure security, and more.
Some use cases require “high touch” data that you need to monitor and analyze continuously or frequently. For example, you need to constantly monitor production applications, troubleshoot issues, and understand your security posture. These use cases require continuous access to data like production web server and application logs; error and warning logs; and compliance and security assurance data.
Other use cases require much less frequent data analysis. Here, we’re talking about “low touch” data that can be very valuable when you want to mine your data for insights, provide periodic reports, or perform a root cause analysis. These use cases can require frequent or infrequent access to data like development, test, and pre-production logs; debug logs; CDN logs; and network logs.
Sumo Logic’s Data Tiers provide a comprehensive solution for all types of data that an organization has, low touch, high touch and everything in between, at an economical price. Data Tiers provide tier-based pricing based on your planned usage of the data you ingest.
Types of Data Tiers
Each Sumo Logic Data Tier supports a different use case and provides its own set of features and capabilities:
- The Continuous tier is for the data you use to monitor and troubleshoot production applications and to ensure the security of your applications.
- The Frequent tier is for data you need to frequently access to troubleshoot and investigate issues. For example, you might use the Frequent tier for development and test data that helps you investigate issues during development. Searching the Frequent tier is free: it's included in the data ingestion price.
- The Infrequent tier is for data that is used to troubleshoot intermittent or hard-to-reproduce issues. For example, you might use the Infrequent Tier for debug logs, OS logs, thread dumps, and so on. The Infrequent Tier has a pay-per-search pricing model, and very low ingestion cost.
* Only supports search visualizations on the log Search page. No dashboard support.
** API support coming soon; will be rate-limited.
Planning your use of Data Tiers
All the data that is ingested into Sumo by default goes to the Continuous Tier, if no other tier has been specified. You use a Sumo Logic Data Stream to assign data to the Frequent or Infrequent Tier. For more information, see Data Tiers and Data Streams below.
When planning your use of Data Tiers, it is important to remember the following guidelines:
- The General Index cannot be changed, and it is always in the Continuous Tier.
- The tier you assign your data to governs how you can search and analyze the data. The table below shows capabilities that are available in each tier.
The amount of data you can ingest to the Frequent or Infrequent Tier is defined by your Sumo account plan. For more information, contact your Sumo Account Representative.
Feature support by tier
How you can search and use your ingested data varies by the Data Tier it resides in, as described in the following table.
|Feature support||Continuous Tier||Frequent Tier||Infrequent Tier|
|Centralized, secure, multi-tenant cloud-native platform|
|Data replication across availability zones, data encryption
Partitions can be specified, but are optional.
Partitions must be specified.
Partitions must be specified
|Field Extraction Rules|
|Logs to Metrics|
|Alerts and View|
How to choose between the Frequent and Infrequent
Choosing between Frequent and Infrequent for a data set depends on how frequently you need to access the data. If you expect to search the data often, the Frequent Tier, with its predictable upfront pricing model, is appropriate. Data that you expect to access less often is an ideal candidate for the Infrequent Tier, which offers low ingest cost, and competitive on-demand search pricing.
For example, for a large development team with hundreds of developers, it is better to send development and test logs to the Frequent Tier if your developers are going to access it often during development.
In contrast, debug or other verbose log sources that are only used to troubleshoot very specific issues that occur infrequently, for example only a couple of times a week, are better off in the Infrequent tier to keep the cost of ownership low.
Please contact your Sumo Logic support representative for more information and guidance in using different Data Tiers.
Data Tiers and Data Streams
You assign data to a Data Tier by defining a Data Stream. A Data Stream specifies a data set, in terms of metadata, key words, and fields, the partition in which to store the data, and the Data Tier where you want the data to reside. For more information, see Data Streams.
Searching the Frequent and Infrequent Data Tiers
When you search for data in the Frequent or Infrequent Tier, you must explicitly reference the partition. For example, to search for errors in a query you must reference the partition by name:
To search more than one partition:
_index=my_freq_partition_name1 or _index=my_freq_partition_name2 error
Common error messages
This section describes the most common error messages for Data Tiers.
If you try to add a panel to a dashboard that uses data from the Frequent or Infrequent tiers, you receive the following error. (You can only use data from the Continuous Tier in a dashboard.)
If you try to specify the scope of a Scheduled View or a Scheduled Search using a partition in the Frequent or Infrequent Data tiers, you receive an error message letting you know that this is not allowed.