Sizing & Configuration Guide for Blue Coat Reporter
This document provides hardware and configuration recommendations for Blue Coat Reporter for various levels of reporting requirements. Blue Coat strongly recommends these sizing recommendations. Also included in this document are recommendations for pruning access logs, setting tuning configurations and adding log filter settings that can significantly increase the performance and scalability of Reporter for large data-sets. All users of Reporter should consider these options and implement them if possible. Reporter should always run on dedicated hardware. Also, for best results always copy logs onto the Reporter server and choose the local log source option in the Profile wizard.
Tables 1 and 2 (below) can be used to determine the hardware specifications recommended based on the size of compressed log files that are/will be periodically added (updated) to the Reporter database and the total amount of compressed logs that are/will be maintained in the database.
The following is an example of how to use Tables 1 and 2 to determine the recommended hardware specifications for the Reporter server
Assumptions: 280MB of compressed logs are generated each day from each of 2 ProxySGs. The Reporter database is updated every other day. A maximum of 30 days of data is maintained in the database at any one time.
Step 1: Determine the amount of compressed logs that are added (updated) to Reporter at any one time.
280MB per ProxySG x 2 ProxySGs x 2 days = 1.1GB
Step 2: Determine the amount of compressed logs that are maintained in the Reporter database.
30 days x 280MB per day per ProxySG x 2 ProxySGs = 16.5GB
Step 3: Refer to Table 1 and determine the appropriate recommended hardware option for these conditions:
The value calculated in Step 1 meets the “IF” (first column) criteria for Options #2 and #3
The value calculated in Step 2 meets the “AND” (second column) criteria only for Option #3
Therefore the hardware specification recommended by Blue Coat is Option #3
Step 4: Refer to Table 2 and see the minimum recommended hardware specifications.
| Option #3 | 4 x Xeon (2.8GHz or faster) | 8GB | External Fibre Channel** | 15k RPM/ RAID 0 | Total Amount of Compressed Logs x 25* | Windows XP and 2003 Servers Red Hat Linux Server 9 |
HP ProLiant DL580 Dell PowerEdge 6650 |
Step 5 :Review and implement the appropriate options for pruning access log fields, tuning Reporter and setting log filters before building the database.
Table 1 – Conditions that are used to determine the recommended hardware option
|
Table 2 – Specifications for each recommended hardware option
| Minimum Recommended Hardware Specifications | |||||||
| CPU | RAM | Storage | Drives | Available Disk Space | Operating System | Example Hardware Models | |
| Option #1 | 1 x P4 (2.8GHz or faster) or Xeon (2.8GHz or faster) | 2GB | Internal SCSI Controller | 15k RPM/ RAID 0 | Total Amount of Compressed Logs x 25* | Windows XP and 2003 Servers, Red Hat Linux Server 9 | HP ProLiant DL360, Dell PowerEdge 1850 |
| Option #2 | 2 x Xeon (2.8GHz or faster) | 4GB | Internal Dual Channel SCSI or External Fibre Channel** | 15k RPM/ RAID 0 | Total Amount of Compressed Logs x 25* | Windows XP and 2003 Servers, Red Hat Linux Server 9 |
HP ProLiant DL380, Dell PowerEdge 2850 |
| Option #3 | 4 x Xeon (2.8GHz or faster) |
8GB | External Fibre Channel** | 15k RPM/ RAID 0 | Total Amount of Compressed Logs x 25* | Windows XP and 2003 Servers, Red Hat Linux Server 9 | HP ProLiant DL580, Dell PowerEdge 6650 |
* Total amount of compressed logs being processed by Reporter for ALL Profiles
** External fibre channel should always be used in combination with a high performance storage system
Access Log Pruning Recommendations
It is important that Reporter customers who generate large log files consider pruning the log fields that are generated by ProxySG. Customers with large log files should log only the fields that are needed in reports.
Reporter Tuning Recommendations
This section provides guidelines for configuring log processing and database tuning options in Reporter to optimize its performance.
This section provides guidelines for configuring log processing and database tuning options in Reporter to optimize its performance.
For customers that meet the requirements for Option #1 (see Table 2 above)
Maintain the default settings
For customers that meet the requirements for Options #2 and #3 (see Table 2 above)
Change the following Database Tuning options:
You can change the database tuning options by Selecting Config (for each Profile) > Select Database > Select Database Tuning > Select Edit Log Processing > Select Edit Database Tuning
For Reporter version 7.1.2.x
- Select the option to “Build all indices simultaneously”
- Select the option to “ Build all cross-references simultaneously”
For Reporter version 7.1.3.x
- Select the option to “Build all indices simultaneously”
- Select the option to “Build indices during log processing”
SPECIAL NOTE: Windows 2003 Server users should set the /3GB switch option to allocate more memory to the Reporter service. Reporter is able take advantage of this option in 7.1.3. Details of how to do this (via the boot.ini file) are at the url below:
http://www.microsoft.com/whdc/system/platform/server/PAE/PAEmem.mspx
Change the following Log Processing options:
You can adjust the log processing options by Selecting Config (for each Profile) > Select Log Data > Select Log Processing > Select Edit Log Processing
Set the “Log processing threads” equal to:
0 (default) for Option #1
2 for Option #2 (for processors without hyper-threading)
3 for Option #2 (for processors with hyper-threading enabled)
4 for Option #3NOTE: For customers using Windows 2000 Server (not recommended), Intel recommends that hyper-threading NOT be enabled.
Log Filtering Recommendations
Log filters can be set in Reporter to reduce the size of the database. For instance, a log filter could be used to reject (exclude) all log entries generated by NTLM authentication prompts. Excluding these entries has the dual benefit of dramatically reducing the size of the database (and consequently the time needed to generate reports) and making the reports easier to interpret (reducing clutter by removing non-valuable entries). It is highly recommended that customers consider the type of information needed in reports and filter out all unneeded entries. Log filters do not alter log files- they only prevent certain data from being added to the database. The following are some non-default log filters in Reporter that should be considered by all customers:
1) Filter out database entries from authentication prompts, responses and redirects (always recommended for customers that use proxy authentication) .
To implement the appropriate filter follow the appropriate steps in options A, B or C below:
A) For customers using NTLM authentication:
Select Config for the appropriate Profile > Select Log Data > Select Log Filters > For “Reject Proxy Auth” Select Edit and Filter Active > Select Save and Close
B) For customers using explicit proxy authentication:
Select Config for the appropriate Profile > Select Log Data > Select Log Filters > For “Reject Proxy Auth (cs_username), Keep Failures” Select Edit and Filter Active > Select Save and Close
C) For customers using transparent proxy authentication:
Select Config for the appropriate Profile > Select Log Data > Select Log Filters > For “Reject Transparent Auth” Select Edit and Filter Active > Select Save and Close
2) Filter out .gif and .jpg entries (recommended for better performance)
Many customers will benefit by filtering out database entries that correspond to graphical images (.jpgs and .gifs). Filtering out these entries will increase the ease of tracking user behavior because all URLs, domains and page views (and their corresponding categories) will still be represented in reports- but the reports will be significantly smaller and easier to manage. This filter should not be used when there is a necessity to track bytes downloaded and hits.
To implement this filter follow these steps:
Select Config for the appropriate Profile > Select Log Data > Select Log Filters > Select New Log Filter > Select Filter Tab > Enter if (file_type eq ‘GIF’) or if (file_type eq ‘JPG’) then “reject”; into the form field > Select Save and Close
3) Reject all non-page views (recommended for customers who just want to track what sites users visited but not the associated hits or bytes)
This is an extension of filtering out common graphical images in #2 (above) in which all non-page entries are excluded from the database. This filter should not be used when there is a necessity to track bytes downloaded and hits.
To implement this filter follow these steps:
Select Config for the appropriate Profile > Select Log Data > Select Log Filters > For “Reject non-page views” Select Edit and Filter Active > Select Save and Close
NOTE: We recommend caution when tracking all, full URLs with Reporter. This is listed as a log filter option but it should be used only with very small data-sets. For larger data-sets it is recommended that you set log filters to isolate a single user if you want to track full URLs.
Buy Now