Kendra Thorpe’s VU Blog

Rebuilt Exchange Servers a multitude of times
Created a SQL 2008 Cluster on Windows Server 2008 Ent
Created a partially functioning Windows Server 2008 Ent cluster for email archive
Build SQL 2008 SQL Reporting Services virtual machine running on Windows Server 2008 for email archive.
Supervised the upgrade of the test ESX Intel cluster to 3.5 update 4.
Scheduled vsphere 4 training

* Created three new SQL Reporting Services reports for the ESX server metrics. I then created three ASPX SharePoint pages with page viewer web parts that reference these reports. I stored the SharePoint ASPX pages in the virtualization SharePoint site so that the entire team has access to the same information.

* I am now the Manager of System Administration in ITS.

* Created Powershell script to automate the collection of VMware statistics.

* Created a Powershell script to send Kevin information on the number of operating systems we have in our environment for physical versus virtual computers.

* Coordinated and built 3 of the 9 new IBM servers for the Exchange project.

1. Rewrote virtual machine inventory script in Powershell – The old script utilized perl, and I rewrote the script using Powershell and the vSphere Power CLI. The data is written to a MySQL database and I utilized the ODBC drivers to connect to the MySQL database.
2. Normalized Runbook data with the NOC inventory data – Due to naming mismatches and servers located in datacenters not managed by the NOC, we had servers that were in the Runbook and not in the NOC inventory. In order to give accurate counts on our inventory, it was necessary to normalize this data.
3. Recreated Weekly Helpdesk ticket in SQL Reporting Services – Previously a sysadmin had to manually calculate the number of helpdesk tickets in the various states. Now the report automatically calculates this information.

SharePoint Implementation – Had a wrap up meeting for the project. We are actually going into production.
Windows Server 2008 Cluster – Still having issues with adding the second node to the cluster.
In my “spare” time – Began work on improving the server provisioning and decommissioning process by adding in workflows for the various tasks and creating helpdesk tickets for work needed by other teams.
SCOM Audible Alerts - Install SCOM, Powershell as well as the scripts that provided the audible alerts on several workstations in the NOC. I am currently using it on my machine without issue.

IBM Director 6.1 Implementation – After moving the new IBM server to the production VLAN, I was able to successfully get several machines to “talk” to the new server. I have hardware inventories for both Windows and Linux machines. I will be putting in a change to put the new server into production sometime in May.

Shibboleth Project – I have had a quiet month this month on the Shibboleth project. I am still waiting on the IDev resource to complete testing on their tasks before I can clone the main virtual machine for DR purposes.

SCOM Audible Notifications – I was able to get SCOM to send audible notifications using Windows Media Player by harnessing the power of PowerShell and PSEXEC. I believe that after further testing, I should be able to put a list of servers in a text file and send the notifications to multiple machines. Once I have the code, I post it here.

SharePoint Rollout – I believe SharePoint is very close to going into production. Some important decisions have been made surrounding this application paving the way for an actual productions date. Whoo Hoo!

OCS Lighthouse – In addition to building one server for this project, I also configured several of the OCS lighthouse servers for teams using LACP. I was able to team two NICs and configure the team for two VLANs. It was a bit frustrating at first, but I’m glad I was able to accomplish this.

• IBM Director 6.1 Implementation – I have successfully installed IBM Director 6.1 on a virtual machine and configured a test machine to communicate with it. I still don’t have all of the information that I did in version 5.20. I still don’t have any controller, BIOS or firmware inventories collected in the new version. I believe that since the system management port has not been configured it can’t report any of that information. I have not had a chance to test the theory yet. Something I wish IBM had included in their documentation that you can download off of their website is the SQL username and password detail. According to the documentation you need a username and password for the IBM Director installation. The problem is they don’t tell you that IBM Director can only perform SQL authentication. That one little detail cost me multiple failed installs, and the installs take upwards of one hour to complete. IBM does document this information in the 900MB iso image file. Thanks, IBM!
• Shibboleth Project – I provisioned two 32-bit Windows for this project as well as various other tasks.
• Windows License Tracking – created a table in a database to keep track of licenses purchased, not just Windows Licenses in ASP.NET.

Team Lead –

• Assist with SCOM Notifications. Our goal is to have SCOM notify specific personnel with an audible notification. There is a way to use PowerShell to open Windows Media Player from a remote computer, and there has been some success on one client computer. We have so far been unsuccessful in reproducing this behavior.
• Assist with Troubleshooting “Document Not Saved” Issue with Office 2007 and Vista – My computer is the only one readily available experiencing this issue. I have worked with my team member as well as sat in on several conference calls with Microsoft to resolve this issue.

Move Runbook to Altiris – Added additional functionality to the Runbook

• Added search page to allow searches by server name or business function.
• Added form to capture last patch date for UNIX/Linux servers
• Added Server and Business function page, then allowed search by server name on that page
• Improved the Data Center disaster recovery report to include IP address information as well as hardware and location information. That report is written in SQL Reporting Services and is automatically sent to our NOC staff on a monthly basis.
• Wrote two reports that capture firmware and BIOS information for IBM servers from the IBM director database

Virtualization Tasks – Successfully relocated and reinstalled two ESX servers to different data centers.

The month of fire fighting
I don’t have much in WINS column because we spent much of the month fighting fires and working on life cycle management.

Fire Fighting 101
Our Exchange 2003 had a significant issue involving mail-enabled Active Directory groups. The root cause was a change that we made to the environment nearly a month prior. The change was to make our empty domain a global catalog server. According to Microsoft this configuration is fine. However, when everyone came back from Christmas vacation we began to see this issue with a vengeance. The kicker was the people sending the messages never received an NDR. Unless the sender either contacted the recipient to verify whether he/she received the message, they would be none the wiser. The fix was to either reconfigure all 1500 of our mail-enabled groups to Universal groups or reconfigure the empty domain. We chose to reconfigure the empty domain.

Life Cycle Management
In this space, I spent much of my time updating reports on the runbook to reflect the correct hardware information. I also added in the building and rack information to the “Altiris OS and Runbook Report”. I also created a SQL Reporting Services report that emails itself to our Network Operations Center. This report will be used in the event we have a disaster in our data centers. The “Data Center Disaster Recovery Report” contains the following information:
OS, Primary Administrator, Purchase Date, Device Type, Service Status, Resource Status, Tier Level, Business Function, Manufacturer, Model, Serial Number, Rack, Position, Number of U’s
We also updated the server/service availability for last year.

SharePoint Rollout – The biggest accomplishment this month was inadvertently “fixing” the SharePoint performance issues we’ve been having. I don’t think I have enough time to list all of the things we have tried and the money we have spent to resolve the performance issues plaguing our SharePoint implementation. So here is how it went down, we are in the process of implementing Systems Center Operations Manager 2007. I installed the agents on the SQL 2005 cluster but SCOM was not able to monitor any of our clusters. I put in a change to configure the SQL cluster computer and service accounts for Kerberos in an effort to allow SCOM to monitor SQL. My team member Roland listed the steps I came up with to configure Kerberos on a SQL cluster in his blog so I won’t repost that information. Monday morning people were complimenting us on how fast SharePoint was running. SharePoint and fast were two words I’d never thought I’d hear in the same sentence. So that’s how we inadvertently fixed our SharePoint performance issues, SCOM! We still have not configured our SharePoint web applications to use Kerberos, they are still using NTLM. However, from SharePoint servers to the SQL cluster is Kerberos. I verified this by running the following SQL script:

select auth_scheme,client_net_address
from sys.dm_exec_connections

This will yield the following results:
KERBEROS 192.168.0.56
KERBEROS 192.168.0.92

Upgrade VCMS to 2.5 – We were successful in upgrading VCMS to version 2.5 this time. The upgrade process took about 15 hours total to complete, so I’m glad that we decided to upgrade the backup server first and copy the upgraded database to the production database server then run the upgrade on the production VCMS server. The upgrade ran on the backup VCMS server took 11.5 hours to complete.

Upgrade ESX Servers to 3.5 – I really didn’t have to do much on this. I simply ran the upgrade script on two our ESX servers.