Mastering Fortinet FortiGate Firewall Logs

Mastering Fortinet FortiGate Firewall Logs - Part 2 Optimization

Observo.ai Team

Overview

FortiGate firewall logs are crucial for network security and compliance. These logs contain valuable information about network traffic, including source and destination IP addresses, ports, protocols, timestamps, and firewall actions. With FortiGate log volumes growing annually, many organizations face challenges in processing and storing these logs efficiently. In part 1 of this series, we covered an overview of Fortigate logs, and some of the challenges they pose for Security and DevOps teams. Here in Part 2, we'll explore how Observo AI optimizes FortiGate logs for cost efficiency and enhanced performance while maintaining their analytical value. Typically, customers who use Observo AI experience an 80%+ reduction in overall log volume for Fortinet logs.

Optimizing FortiGate Logs

Let’s take a look at some examples showing different ways to study Fortigate logs.

Analytics

*Figure 1: Breakdown Fortigate logs based on various dimensions. In this figure, we see a distribution of Device Information across all our Fortigate logs.*

‍

*Figure 2: Breakdown Fortigate logs based on Interface IDs. Many organizations have interface IDs which are “trusted” and hence these logs do not add value in downstream SIEMs.*

‍

*Figure 3: Breakdown Fortigate logs based on Log Severity. It’s likely that debug and some informational logs are not useful to your organization.*

When designing a data pipeline, understanding the nature of your data is crucial. Observo AI addresses this need with its integrated stream analyzer, offering users a comprehensive overview of their dataset and its temporal evolution. For Fortinet FortiGate logs, this tool provides multifaceted insights. As illustrated in the example, users can view the distribution of FortiGate device IDs. The analyzer also allows for flexible data segmentation across various dimensions such as OutboundInterface and LogSeverity. These analytical capabilities are fundamental in guiding informed decisions about data management and processing strategies.

Aggregate Noisy Traffic Logs

FortiGate Traffic logs can be particularly noisy due to their high granularity and volume. These logs are generated for every network session passing through the FortiGate firewall, resulting in an enormous number of entries, especially in busy networks. The sheer volume of these logs not only consumes substantial storage resources but also increases the computational overhead required for processing and analysis. Consequently, important security events or anomalies might be obscured by the noise of normal traffic patterns, making it difficult for administrators to effectively monitor and respond to potential threats or performance issues in real-time.

Observo AI can significantly help aggregate FortiGate traffic logs, reducing log volume while maintaining critical information. Here's an explanation with examples:

FortiGate traffic logs can be extremely verbose, generating an entry for each network session. Observo AI's reduction engine consolidates these repetitive logs within condensed time frames.

Without Observo AI

With Observo AI:

In this example, Observo AI has aggregated multiple log entries from the same source to the same destination over a short time period. The aggregated logs include:

A time range (start and end time)
Total bytes transferred
Number of sessions

This aggregation significantly reduces the number of log entries while preserving essential information about traffic patterns. It can lead to over 60% reduction in log volumes, optimizing storage and improving SIEM system performance by reducing the amount of data that needs to be processed and analyzed

Sample Reserved Logs

Observo AI can sample Reserved FortiGate logs to reduce unnecessary data volume while preserving critical information. Reserved logs, often generated within trusted zones of the network, typically contain data from internal communications presumed to be secure and less likely to pose security threats. By implementing customizable sampling and filtering strategies, Observo AI can selectively forward only high-value, meaningful information from these logs to downstream systems.

Filter Out Link Local Traffic

Observo AI can significantly reduce log volume by filtering out link-local traffic logs from FortiGate firewalls. Link-local traffic, which occurs between devices on the same network segment (typically using IPv6 addresses in the fe80::/10 range or IPv4 addresses in the 169.254.0.0/16 range), is often less relevant for security analysis and can generate a substantial amount of noise in firewall logs. By implementing intelligent filtering rules, Observo AI can identify and exclude these link-local communications from being forwarded to SIEM systems or long-term storage. ‍

Multi Destination Routing

Observo AI enhances the management of FortiGate logs by enabling multi-destination routing, ensuring that different log categories are directed to the most appropriate destinations for processing and storage. For instance, security logs documenting attempted breaches and malware activities are routed to SIEM system for immediate analysis, while traffic logs detailing network patterns are directed to cost-effective storage for long-term analysis. System error logs, indicating technical issues, can be routed to an IT operations management system for troubleshooting. This targeted approach optimizes log management, reduces unnecessary storage costs, and enhances the performance of security and IT operations systems.

Typical FortiGate Log Optimization Results

Table 1: Captures average optimizations we have seen across customers and is a good representation of optimizations a typical customer can expect. The optimization numbers presented are for each individual transform and need to be applied cumulatively to get overall optimization.

Conclusion

FortiGate firewall logs are essential for network security, but their volume can strain analytics systems and budgets. Observo.ai's solutions for FortiGate logs include:

Generate insights and summaries on Fortigate logs in order to make decisions when building a data pipeline.
Dramatically reducing log volume through aggregation
Summarizing repetitive logs
Dynamic sampling for further data reduction
Multi-destination routing

These optimizations help organizations manage their FortiGate logs more efficiently, reducing costs while maintaining the ability to detect and respond to security threats effectively.

Stay tuned for part 3 of this series, where we'll explore a customer use case demonstrating Observo AI’s effectiveness in optimizing FortiGate logs in a real-world scenario.