1) QoE and CDR Database [...]]]> The Call Detail Record (CDR) and Quality of Experience (QoE) data available through the OCS 2007 R2 Monitoring Server role enables very good usage and quality of experience reporting.  This blog post covers how to estimate the size of the database, and how to manage the database data rentention and purge time.

1) QoE and CDR Database Sizing
Based on a sample Microsoft OCS feature usage model (the same one used for the OCS 2007 R2 Capacity Planning), the CDR database grows 8.8 KB per day, and the QoE database grows 16.8 KB per day.  Once we know this, we can estimate the database size using this formula:

> Database size (KB) = (DB growth per user per day) * (Number of users) * (Number of days)

For example, the CDR and QoE database in a 3500 OCS user organization would consume a total of 5.3 GB in 60 days ((8.8+16.8) * 3500 * 60).

2) Setting the Data Retention
The default data retention for both CDR and QoE data is 60 days. Every night the purge process will delete any records which are older than this value. The retention setting can be viewed and set in the OCS Administrator Console by doing a right-click | Properties on particular monitoring server in the left-hand tree. The property page looks like the following:

Set the Maximum Days Logged to the number of days that you wish to retain the respective data sets.

Note:

• To purge everything, use a value of zero.
• Leave the checkbox unchecked to turn-off purging all together.
• You can also manage these settings through WMI.

3) Setting the Purge Time

Specifying the time of day that the SQL database purges the data cannot be set through the OCS Administrative Console. You need to use WMI to make this setting. The WMI classes to set the QoE and CDR purge times are shown below:

1. QoE:  use the MSFT_SIPQoESetting class and set the PurgeJobStartTime property.  The value is an integer representing the number of hours after midnight to start the purging (i.e the hour based on a 24 hour clock).  For example, a value of “1” will start the purging at 1am.
2. CDR:  use the MSFT_SIPCDRServiceSetting class and set the PurgeTime property. This also is an integer value representing the number of hours after midnight to start the purging of outdated records.

These classes are only installed and available on the Monitoring Server. However, you can use the “-computername” parameter in WMI provider to remotely view and set value.

For example, the following two lines of Powershell, can be used to remotely set the Purge Time for the CDR database to be 4am:

Note: if you are using multiple monitoring servers connected to one monitoring database, it is important to not start the purging on each server simultaneously.  See the TechNet article Microsoft Office Communications Server 2007 R2 Configuring Retention of QoE and CDR Data for more information

Addition Information

• Microsoft Office Communications Server 2007 R2 Configuring Retention of QoE and CDR Data (TechNet)
• Key high-level information about the OCS 2007 R2 Monitoring role: OCS 2007 R2 Monitoring Role FAQ (InsideOCS post)
• Microsoft Office Communications Server 2007 R2Monitoring Support (TechNet)
• MSDN MSFT_SIPQoESetting (New)
• MSDN MSFT_SIPCDRServiceSetting (New)

Note: If you are deploying the OCS 2007 R2 Monitoring Role, there is an updated (Jan 2010) post with answers to key questions about deployment: see OCS 2007 R2 Monitoring Role FAQ.

1. [...]]]> While preparing my last post, I consolidated some quick facts about the QoE functionality that should benefit people who are considering deploying this role:

Note: If you are deploying the OCS 2007 R2 Monitoring Role, there is an updated (Jan 2010) post with answers to key questions about deployment: see OCS 2007 R2 Monitoring Role FAQ.

1. There is no additional server license (to your enterprise or standard license) required for this role. For OCS 2007, the role can be downloaded here.

2. In OCS 2007 R2, the QoE functionality and the Call Detail Record (CDR) functionality were merged into the OCS 2007 R2 Monitoring Role.

3. Supported OCS 2007 QoE Collocation:
- The OCS 2007 QoE Monitoring Server cannot be collocated with other Office Communications Server 2007 server roles.
- The OCS 2007 QoE SQL Database can be collocated with the QoE server, or installed on a dedicated server.

4. Supported OCS 2007 R2 Monitoring Role Collocation:
- A variety of collocation scenarios are supported depending on your scalability requirements.
- The OCS 2007 R2 Monitoring Role service and database can be collocated with a computer running Standard Edition (very small deployments only). If you do this, the full edition of SQL Server must be installed on the server (instead of the SQL Server Express Edition that is normally used).
- The OCS 2007 R2 Monitoring Role cannot be collocated with a Consolidated Enterprise Edition.
- It can be collocated with other individual roles such as the Archiving Server.

5. It can serve multiple pools.

6. A certificate is required on the QoE server, so that other OCS servers can use MTLS to communicate with it. This can be a self-signed certificate.

7. For QoE database sizing, the Office Communications Server 2007 Quality of Experience (QoE) Monitoring Server Guide has a formula that helps you size your database depending on retention settings and expected volumes.

8. Key software requirements:
- 2005 Reporting Services SP1 or SP2 on the backend QoE / Monitoring role SQL database to get reports
- For the OCS R2 Monitoring Role, you need to install the optional Report Pack for Monitoring server component.
- Message Queuing must be installed on the computer that will run the QoE Monitoring Server.

9. Basic Hardware Requirements for OCS 2007 QoE:
- a+2.6 Ghz dual processor with 4 Gb RAM
- a 1 Gb network adapter
- a 2 x 18 Gb disk for O/S and Page File
- a 2 x 36 Gb disks for database data and database log files

10. There is one known issue if you have an R2 pool and an OCS 2007 pool. From the OCS 2007 R2 release notes:

Office Communications Server 2007 R2 Pools Display Incorrectly in the Office Communications Server 2007 QoE Monitoring Server MMC Snap-in: “After installing Office Communications Server 2007 R2 Standard Edition or Enterprise Edition pool, the Office Communications Server 2007 QoE Monitoring Server MMC snap-in incorrectly shows the new pool as being a QoE Monitoring Server. If you try to connect through the snap-in, an error message is display. Resolution: You can safely ignore any pools that show up as a QoE Monitoring Server in the Office Communications Server 2007 QoE Monitoring Server MMC snap-in.

11. The QoE is kept by default for 60 days.

12. The Microsoft OCS QoE protocol specification is available here: http://download.microsoft.com/download/1/6/F/16F4E321-AA6B-4FA3-8AD3-E94C895A3C97/[MS-QoE].PDF.

13. See How to Manage the OCS 2007 R2 QoE and CDR Data post for how to size the database and set the retention and purge times.

How the QoE data is Collected (high level)

1. QoE metrics are collected at the end of every VoIP and Video call from participant endpoints (including Office Communicator, Office Live Meeting, the A/V Conferencing Server, Mediation Servers and IP phones). The metrics are collected and sent in a SIP header to the OCS Front-End server by each client.

2. A QoE agent runs on each Front-End server. The agent then uses Microsoft Message Queuing (MSMQ) to transmit the data to the QoE or Monitoring Server. These agents are installed automatically on every Front End Server and Standard Edition server. They are activated automatically but no data is recorded unless a Monitoring Server is deployed and associated with that Enterprise pool or Standard Edition server.

3. The Monitoring Server takes the information from the queue and puts it into the SQL database.

Additional Links:

Office Communications Server 2007 Quality of Experience (QoE) Monitoring Server Guide

Office Communications Server 2007 Quality of Experience (QoE) Monitoring Server Audio and Video Metrics Processing Guide

Office Communications Server 2007 Document: Archiving and CDR Server Deployment Guide

Office Communications Server 2007 Document: Planning Guide

Microsoft TechNet – Archiving and Monitoring for Office Communications Server 2007 R2

Below is a quick summary of the most available options:

1) [...]]]> Getting insight into how your OCS deployment is being used is important for planning, maintaining, troubleshooting, and justifying the ROI. There are a wide variety of reporting options – each with it’s own niche – everything from free standalone resource kit tools to 3^rd party solutions.

Below is a quick summary of the most available options:

1) Administrative Console Reporting
Embedded directly in the OCS Administrative Console are some high level basic usage reports at the pool level. Select the Database tab on the pool and there are several reports that are available: User Summary, Per-User, and Conference Summary reports. These reports are limited in scope and depth, and only provide a current snapshot of usage – good for troubleshooting, but it does not provide deep insight into how your OCS deployment is being used.

2) OCS Resource Kit Tools
The Office Communications Server 2007 Resource Kit Tools provide three rudimentary utilities that provide basic usage reporting.

a) ArchivingCDR Reporter: is a standalone Win32 exe with a GUI that issues SQL queries to the OCS database to give you the following basic usage reports:

b) DbAnalyze: is a command line tool with 5 different modes; 3 of them are applicable provide usage reports:

1. User data mode. Reports contact, container, subscription, publication, permission, and contact-group data for a specified user.
2. Conference mode. Reports on specific conference details: scheduled-time, the invitee list, the list of media types allowed for the conference, active MCUs, the active participant list, and each participant’s signaling state.
3. MCUs mode. Reports the ID, media type, URL, heartbeat status, conference load, and participant load for each MCU (multipoint control unit) in the pool.

All data is output in text to the command line, but it can be redirected to a file.

c) Public Internet Connectivity Usage Query (PICStats.sql)
PICStats.sql are example SQL queries that can be used against the OCS 2007 ‘rtc’ backend database, and it provides statistics related to public IM connectivity usage. You will require a command line utility like osql.exe to run the sample SQL scripts.

The information returned by the SQL queries:

1. The min, max, average, and standard deviation for the number of AOL, Yahoo!, and MSN Contacts for all users.
2. The min, max, average, and standard deviation of the percentage of a user’s contacts that are PIC contacts (for users with at least one PIC contact).

3) OCS 2007 Quality of Experience Role (QoE) /  OCS 2007 R2 Monitoring Role
Note: starting in OCS 2007 R2, the QoE and Call Detail Record (CDR) data and functionality was moved into the OCS R2 Monitoring Role.

The QoE functionality collects VoIP call quality metrics from the participant endpoints including Communicator, Live Meeting, the OCS A/V Conferencing Server, and Mediation role. Quality reports can then viewed on these metrics, giving you insight into the media quality in specific network locations, view quality trends, and perform diagnostics. The other significant use for this data, is to alert on quality issues.

The available OCS 2007 QoE reports are focused on quality of VoIP calls, and not the overall OCS feature usage or trending, so these are best used to gain insight into the quality of VoIP calls (including their use in peer-to-peer, Conferences, Live Meetings, PSTN integration). With the OCS 2007 R2 Monitoring role and the Monitoring Report pack, there are 3 additional new Call Detail Record (CDR) reports which provide basic peer-to-peer and conferencing usage information. This is addition to the 9 media quality reports.

For more information on the types of reports available, and how to deploy the OCS 2007 QoE role, see Microsoft Office Communications Server 2007 Quality of Experience (QoE) Monitoring Server Guide.

Note: you need to install SQL 2005 Reporting Services on the backend QoE / Monitoring role SQL database to obtain these reports, and for OCS R2, you need to install the optional Report Pack for Monitoring server component.

4) Quest MessageStats
Quest MessageStats™ is a complete and comprehensive messaging reporting and analysis solution for Microsoft Exchange, Blackberry, and other platforms. A report pack was recently added for Office Communications Server 2007 that provides insightful and easy-to-use reports for all aspects of the OCS usage, including instant messaging, VoIP, audio and video conferencing. The big advantage of a solution like this is the ability to do trend analysis over time.

5) Microsoft MOM/SCOM Management Packs
The Office Communications Server 2007 QoE Monitoring Server Management Pack for MOM 2005 is focused on monitoring, but as with most management packs, reports are included for most of the QoE data, which includes some usage reporting. Likewise, a Office Communications Server 2007 R2 Management Pack is available for System Center Operations Manager 2007 SP1 if you are running OCS R2.