It is important to monitor the size of the NVMonitorData SQL database that is used by StealthINTERCEPT (SI) to store the event data it collects. In production environments, the event dataset can grow significantly over time. If left unchecked this DB growth will lead to excessive disk space usage and slowing performance over time inserting new event data. In addition, users can encounter slow performance reporting data via either the SI Console or the Web Reporting modules.
To assist with managing the data stored in the SQL DB the StealthINTERCEPT product includes a “DB maintenance” system to regularly groom the DB content. It supports a two-tier storage model. Tier one or the ‘production’ DB is the SQL DB specified when the SI product was installed. This is where all data collected in real-time is stored and used for reporting including “Investigate”, “recent events” and “Web reports”. An optional tier two DB called the “Archive DB” can be defined in the Database Maintenance dialog “Archive DB” tab. In the SI Console ‘Investigate’ panel users can toggle between the “Production” and “Archive” databases when searching for event data.
The “Database Maintenance” dialog accessed from SI Console Configuration | Database | Maintenance menu is s GUI front end. It configures a SQL Server jog via the SQL Server Agent to run a series of stored procedures either deletes or moves SI event records based on the age of the event and is run at a specified time either once, daily, weekly or monthly.
Our recommended best practices include the following:
It is currently up to the customer to manage content in the Archive DB to limit size. One approach is to periodically delete all events older than a specified date using native SQL queries.
Another is to periodically (i.e. yearly) create and point the SI DB Maintenance feature to a new Archive DB. Keep the ‘old’ Archive DB as is but ‘offline’ relative to SI. If you need to later get at data in one of these ‘old’ DB’s it can be done in one of three ways:
Assuming you don’t want to drop the DB and start over… If you never used DB Maintenance but want to after the production DB has become huge (i.e. many TB) you may find that attempts to move or delete events can run for days or even never finish. In this case, one may have to ‘ease’ into it. By that, we mean to configure the type tab picking every type and set to retain data at first for only a few days less than your oldest event. For example, if you oldest event data is 700 days then set ‘retention period’ to 695 days. Then run DB Maintenance either that night or ‘now’. This first run is more about verifying the job runs w/o errors than deleting any real amount of old data. Take note of how long the job runs (look at DB Maintenance start/end alerts in SI Console). If all ran well then try setting ‘retention period’ to 10 days less than the now oldest event and repeat looking at total job run time. Find a ‘retention period’ value relative to the now oldest event that takes less than say 6 hours to complete. Repeat until you are able to get down to the desired “retention Period”. Then set DB Maintenance to run nightly so thereafter you never have events older than your ‘longest’ retention period value by Type or policy left in the production DB.
Anthony Sarra is the VP of Research & Development focused on building and delivering real-time activity solutions for Stealthbits – Now part of Netwrix including the StealthINTERCEPT and Activity Monitor products. He joined Stealthbits in 2010 as part of the acquisition of NetVision Software.
Anthony has 40 years of experience in software development and management roles creating enterprise software solutions ranging from factory automation to desktop management and server security. Prior to NetVision and Stealthbits Anthony held software developer and leadership roles at Intel Corp. and Intel spin out LANDesk Software.
Proper data security begins with a strong foundation. Find out what you're standing on with a free deep-dive into the security of your Structured and Unstructured Data, Active Directory, and Windows infrastructure.
Read more© 2022 Stealthbits Technologies, Inc.
Leave a Reply