Archiving logs is not a straight forward process. Storage administrators have to balance the regulatory requirement to archive logs, the data analytic needs, and the cost of long-term retention in a digital landscape that’s constantly threatened by ransomware attacks – calling it a challenge is an understatement.
Not to mention, archive systems aren’t exactly built for efficient power consumption which adds to the challenge especially for organizations with limited resources.
But why do you have to archive logs? What challenges to expect? And what features can make log archiving solutions affordable, secure, and reliable? In this blog we answer these questions, and more.
Why archive your logs?
From event logs to network access to user actions, logs contain records and provide vital information which makes it important for businesses to analyze and store them. Even a small businesses’ IT system can generate terabytes of log data per day – adding up to several terabytes per month. In order to effectively manage these logs and ensure cost-effective retention, for compliance and data analytics, log archiving is necessary.
Log archiving enables storage administrators to free up storage capacity and hardware resources of production infrastructure in addition to making it easier to distinguish older log files from the recent ones.
- Comply with regulatory standards – For forensic analysis, compliance regulations may require you to archive logs for several months, to a year, or more depending on the regulation and industry.
- Facilitate data analytics – Logs contain important information about user behavior, and events. Archive logs enable you to analyze more information and identify trends and patterns accurately.
- Optimize storage consumption – Instead of storing logs on your production and consume storage capacity, it’s more optimal to use purpose-built log archiving appliances. Not only does this free-up resources for your production workloads but is also more cost-effective.
Do you need to secure log archives?
Traditional log archiving systems are built to focus on affordable long-term retention which is why most storage administrators use unreliable and insecure storage hardware such as tape arrays. This approach is costly in terms of time and resources, and risks business IT systems by being vulnerable to ransomware attacks.
While a slow recovery speed is acceptable for an archive system, data breach or data loss is not. Moreover, since traditional archive systems are connected to the production network, a vulnerability in an archive system essentially risks the entire network because ransomware uses the network accessibility to spread to the production, connected storage devices, and backup server(s).
Ransomware attacks grow increasingly complex. They are programmed to find and exploit vulnerabilities in an organization’s network. Without ransomware protection capabilities, archive systems, whether they use tape or disk, provide cybercriminals the access they need to infiltrate the network. This is why it’s necessary to secure log archiving systems with automated ransomware-proof data protection features.
Log Archiving Challenges
Secure Log Archives from Ransomware Attacks
Log archives contain vital information and are typically network-facing which makes them easy targets from ransomware attacks looking for vulnerable access to the corporate network. A vulnerable log archiving solution risks not just the log archives but also the production, connected storage devices, and backup server(s). This makes it necessary for log archiving solutions to make sure that the archives aren’t just affordable but also secure.
Power consumption, heat, and carbon emissions – when the archiving system is idle
Traditional log archive systems aren’t optimized for efficient power consumption, heat generation, and carbon emissions when the system is idle. This means that even if the archives are not being read/written, the log archive system continues to consume power, generate heat, and carbon. Not only is this bad for the environment but it’s also costly for the user.
Alternatively, tape-based archive systems, have to be managed and powered-off manually which is inefficient, error-prone and dependent on the dedicated IT staff. What happens when the storage administrator is sick, or unavailable?
Large volumes of archive logs and increasing growth of log data
Every device in the network generates logs – adding up to several megabytes to terabytes per day. The log archiving system needs to be easy-to-scale so that it can keep up with the growth without disruption.
Maintaining log archiving infrastructure
As traditional archive systems aren’t powered-off when idle, the hardware components such as the drives, power supply, etc. fail from time-to-time and have to repaired/replaced. This increases operational expenses (OpEx) and impacts the total cost of ownership (TCO).
Log archive management
Log archive systems, depending on the storage capacity, consist of a number of RAID arrays, storage server(s), and/or blades. This makes it a challenge to manage not just the archive logs but also the hardware. It takes more time, a steep learning curve, and dedicated IT staff to make sure that the archives operate seamlessly.
Regardless of the industry or the scale of the business, log archive systems need to be easy-to-manage so that they don’t take from the budget that’s meant for the core business operations.
Must-have features in a Log Archiving Solution
Secure archive logs with automated air-gap and immutability
Log archive security should be seamless, affordable, and should not require constant management which is why the best way to do it is to use automated air-gap and immutability.
Air-gapping isolates the archive logs from the production network. This protects them from threats that are able to infiltrate the network by other means, specifically ransomware attacks. It’s important to note that air-gapping when done manually, as such in the case of tape arrays, defeats the purpose as it is prone to human-error, requires constant management, and dependent on the availability of the dedicated IT team.
Immutability makes sure that the archive logs cannot be edited, overwritten, or deleted. This protects them from the malicious encryption of ransomware and helps with compliance.
Low Power, carbon emissions, and heat generation
Log archiving systems need to be energy-efficient. This is necessary for the environment and helpful for the business. By reducing the power consumed per terabyte, the log archiving infrastructure generates less heat. This reduces the need for air-conditioning and the resulting carbon emissions. And less air-conditioning means reduced operational expenses.
The best way to reduce power consumption is by having the log archiving infrastructure powered-on only when the archive logs are being read/written. However, this has to be automated or else it is error-prone and inefficient.
Low maintenance log archive infrastructure
Moreover, by powering off the archive nodes, the need for maintenance is also reduced because the hardware is not accessed frequently. This implies that they are powered-off most of the time. As a result, the hardware is less prone to failure and requires less maintenance.
Highly scalable – no forklift upgrades
The log archiving infrastructure must be highly scalable and should be able to scale without disruption. Scalability makes sure that users can choose to start small, pay less upfront (CapEx), and then scale the archive storage as the log data grows.
How to build your log archive system?
Planning and building a log archive system can be a challenge. Before you begin to think of the hardware, you need to analyze your archive logs and determine:
- The size of log archives per month.
- How long you need to retain the logs (months, a year, or more?)
- Access frequency of the log archives
- How quickly you need to retrieve log archives – when necessary.
This information will help you choose the best log archiving solution for your requirements and budget. When choosing a log archiving appliance, it’s helpful to know:
- How much storage space should the archiving system have to start with?
- If you’ll be archiving your logs on-premises, in the cloud, or building a hybrid log archive solution (on-prem + cloud).
Archive your Logs with StoneFly Green Petabyte Archive (GPA) 4.0
StoneFly GPA 4.0 is 1000W low power, low carbon, low heat, scalable archive system with automated air-gap and immutability.
The GPA 4.0 architecture comprises of two major components: master controller and air-gapped nodes. The master controller is network-facing and connects with your network while the air-gapped nodes are isolated and only powered-on when the log archives are being read/written.
Only the master controller has access to the air-gapped nodes. The master controller powers the air-gapped nodes on and off using a built-in network and power management controller – as per user-defined policies. As log archives aren’t frequently read or written, the air-gapped nodes are powered-off most of the time. This makes GPA 4.0 energy-efficient producing less heat and carbon emissions in comparison to traditional archive systems.
The master controller chassis consumes 220W power and each 4U rackmount air-gapped node chassis consumes 770W – making the total power consumption less than 1000W per petabyte.
The GPA 4.0 comes preconfigured with StoneFly’s patented storage virtualization engine StoneFusion™ that supports iSCSI and Fibre Channel SAN, NFS and CIFS/SMB NAS, and S3 storage protocols.
What does 4.0 stand for?
The 4.0 is a standard for archive systems, and it stands for:
1 – Data security to protect archives from ransomware attacks
2 – Low power consumption and carbon emissions (1000W per petabyte)
3 – Low heat
4 – Low maintenance
0 – Zero ransomware incidents (Zero trust)
Conclusion
Log archiving help data analytics – enabling organizations to get more out of their data and make data-based decisions to improve marketing and overall productivity. The best way to archive logs is to use a log archiving solution that automates network and power management and is ransomware-proof, energy-efficient, low carbon, low heat, and low maintenance.
StoneFly GPA 4.0 makes log archiving affordable, secure, and seamless for organizations of all sizes and all industries. Talk to our experts to custom-build your log archiving solution.
