Tony’s Place

EXCHANGE
Exchange CAS Build of ITS-HCWNEM77
Built new Exchange CAS in support of the MC FE collapse.
Troubleshoot Admin Network issues on ITS-HCWNEM76
Troubleshooting of intermittent admin network connectivity. It appeared to be a conflicting IP address but couldn’t get a ping back from another machine when this machine didn’t have the address configured. The issue ended up being a Cyclades switch that wasn’t responding to pings. Fixed the issue with a new ip on its-hcwnem76.
Assisted in testing MC exchange FE collapse
Helped the exchange team by taking part in the testing of the new MC Exchange CAS environment after the collapse.
Pushed for identification of Exchange Local FW rules for the new MC Exchange CAS’s
Windows Patches of UM environment
Patched the UM environment and troubleshoot resolve issue with ITS-HCWNEM75 CAS that was related to ASP.NET 1.1
Resolve SCOM Agent issues on Exchange boxes
Reinstalled SCOM agents after identifying local agent issues on EM01,EM04,EM75,EM76 and EM77
Memory Upgrade of 2007 Mailbox Servers
Upgraded memory on EM01-EM04 to 48Gb
Exchange Mailbox Switch Redundancy
Identified issues with mailbox servers teamed network switch redundancy. Documented the issues and put in changes to resolve them where possible.

WINDOWS
VUSpaceGroups Migration Project
Started documenting requirements, issues and procedure for migrating VUSpacegroups from Windows cluster to NAS.

*WINS1 Replacement
Replaced WINS1 production server with like replacement to resolve problems that were causing issues with our environment. Identified configuration difference between old WINS1 and WINS2 that was rectified in the replacement process. Documented replacement process.
Pertrac
Assist in troubleshooting/resolving issue with test pertrac server. Be available for when Pertrac needs connection to servers.
PRT1 replacement
Continued work on PRT1 replacement. All print queues have been created on the new print server minus the printers that the server does not have connectivity to. Working with Partner to identify LSP’s and contact them about the printers with no connectivity to ensure that they are not needed anymore to allow cleanup of them from the new server.

Antivirus Repository
Local testing of new Antivirus Repository has run into some issues that we are trying to identify the root cause of.
SCOM Rules
Identified issues with some of the custom SCOM rules that we had in place and reconfigured/documented the process for correctly limiting SCOM rules to a specific group or box.

PowerShell Scripting
Ping Test Script
Wrote a script to test ping connectivity. This was used to identify printers that were configured on PRT1 that had no connectivity.
Printer Information Script
Wrote a script to pull all the relevant information for each Print Queue remotely from PRT1 to facilitate the migration process to a new print server.
Assisted Roland with build doc automation script
Helped Roland with troubleshooting errors he was seeing in his base build doc automation script.

Identified and resolved WINS connectivity issue with NCS Wins server
The issue appeared to be related to WINS server rename by the medcenter. Removal and readd of the WINS partner with correct name seemed to resolve the issue.

Resolve WINS database corruption
Identified wins corruption issue and executed break fix to bring it back online in a timely manner. Subsequently identified event viewer messages to key off of for SCOM and created new Crit/High alert that would appear in the NOC view of SCOM for notification.

Assisted in security investigation of WINS1

Decommission of EMC’s Replication Manager and Replistor
Decommision of RM server and uninstall of RM agents and Replistor software from Exchange 2003 Test and former production environments.

Patching of UM environment
Microsoft patching of the full messaging environment

Page file upate of UM environment
Standardized page file setup across the messaging environment servers.

Participated in buildout of new Exchange Machines in timely manner
Facilitating the build of the Exchange environment by building the new servers and ensuring consistency of the not only the software builds but the network setup of the trunked lines to minimize possible issues. Facilitated fixing of any network issues/differences found with the boxes.

Identified and resolved BIOS memory issue on rebuild of Exchange boxes
Identified BIOS issue on ITS-HCWNEM10,11 and 12 for adding of memory for re-use of these boxes. Installed BIOS and verified functionality of the servers.

Assisted in reworking of OWA rich web monitor conversion to work with Exchange 2007 environment
Assisted Peter in troubleshooting the old Exchange 2003 rich web monitor conversion to work with the Exchange 2007 environment.

Participated in Exchange Monitoring setup
Participated in meeting to ensure that monitoring of the current Exchange environment was satisfactory.

Troubleshot and resolved phantom drive issue on ITS-HCWNEM01,02 and 04
Worked with EMC to identify the issue that was causing the phantom drives to show up on these boxes. The issue was found to be the boxes fiber registration with the CX-340 without having any luns presented from that box. This caused the phantom drives to appear. The issue could be resolved by either presenting storage from that san or unregistering the boxes from that particular SAN. The latter was done leaving storage being presented just from the CX-380.

Cleanup of SCOM DB and change of default size/growth
Ran cleanup scripts and requested scheduling of these scripts on a weekly basis to keep database in a healthy state. Running of the scripts to cleanup the localizedtext tables keeps SCOM running more efficiently. We also increased the starting size of the database and increased the auto-growth to resolve size reporting issues of the SCOM db.

Identified issue with SCOM maintenance mode not working as desired
SCOM maintenance mode was not stopping all alerts from the associated objects with the client. It was only stopping alerts directly from the agent on the box. Found a powershell script that gives a gui tool to put a computer and all associated objects with that box in maintenance mode so no alerts are generated for it.

Put together SCOM documentation for NOC consumption
This documentation was generated for identifying how SCOM works. What the differences are between SCOM and Nagios, and also how to utilize the console to maximize our gain from it.

18 Closed Magic Tickets

Exchange
5 Migrations to Exchange 2007

Windows

VUGroupspace
Identified issue with quota software being utilized to manage VUGroupspace. One of the shares is being reported as uitlizing much more space then is actually in use, as reported by Windows. The issue appears to be a bug in the Quota software. Change has been finished and testing is continuing.

Exchange
Assisted in troubleshooting and resolving various issues this month.

Archiving
Assisted in the process of choosing an archiving solution for the Exchange 2007 Environment. This has included attending various vendor presentations, providing system configuration information and challenging\questioning vendor configurations and abilities.

Cross Training
Continued cross training new messaging employees with current configuration, setup and procedures for the VU Exchange Environment.

Exchange 2007

OWA Vanderbilt Rebranding work
Passed on knowledge for the rebranding for management to Antwan on the Email team. Assisted in putting new logo on OWA Site.
F5 OWA traffic rules
Identified solution for F5 forwarding of vunetid url to delivery address url. This allows for less impact on users after the OWA consolidation for those using macs and those accessing resource URLs.

Exchange integration effort
Worked as the exchange resource for identifying current tools and desired tools for migration to the Exchange 2007 environment and ongoing tool requirements.

PowerShell Scripting

OCS User Enabling
Scripted solution automate the setting up of users for utilizing OCS. This saves considerable time when going through this process.

Cluster Resource Script
Scripted the stopping and starting of cluster resources. Still working through the permissions to be able to run the script remotely from SCOM.

22 Closed Magic Tickets

Exchange
38 Migrations to Exchange 2007

Windows

VUGroupspace
Identified issue with quota software being utilized to manage VUGroupspace. One of the shares is being reported as uitlizing much more space then is actually in use, as reported by Windows. The issue appears to be a bug in the Quota software. Change is in place to update the software to the latest rev.

Exchange

Archiving
Assisted in the process of choosing an archiving solution for the Exchange 2007 Environment. This has included attending various vendor presentations, providing system configuration information and challenging\questioning vendor configurations and abilities.

Cross Training
Assist in cross training new messaging employees with current configuration, setup and procedures for the VU Exchange Environment.

Exchange 2007

OWA Vanderbilt Rebranding work
Installed new test\production CAS for internal testing in the production environment before implementation. This CAS will also be utilized for testing OWA integration.
Finished the OWA Rebranding and put it into production. Continuing the documentation of the .css files on my time for easier modification in the future if the need arises. Modified the test CAS setup to allow for multiple images at the top of the OWA login page to accomodate MC and VU logo’s for the combined environment.

OWA timeout issue
Worked with Peter in identifying the issue that was causing our 2007 OWA timeout issues. Tested solution on the test/production CAS successfully. Peter is putting in a change for implementing ths change on the production OWA Load balanced ip.

OWA integration effort
Worked on identifying issue that is preventing OWA 2007 from connection to VUExchange 2003 mailboxes. Still have not identified the complete root cause, it appears as if it is because the default internal delivery address for the VUExchange users is different from the default proxy address for the Exchange Enterprise. It is still being researched as to why this is happening

OWA 2007 Change Password
Identified the method for disabling the change password option for OWA 2007. This is important so as to not cause confustion issues with end users since this functionality will not work since AD is not authoritative for passwords and current ad permissions do not allow for changing the password in this fashion.

UM
Identified issue with loader process overwriting UM and OCS proxy address information in AD.

PowerShell Scripting

Mailbox Migration
Scripted to help automate the process for larger scale migrations. The tool automatically picks the storage group with the least number of mailboxes currently on it and creates a mailbox move script for moving each user that is listed in the users.txt file that is populated with the users to be migrated from 2003 to 2007. The next portion of the script that I am working on is automating the start of each of the created user move scripts. The tool will run up to 10 migrations at a time and then prompt for the script to be run again. It keeps a list of all the users that were moved and the dates. Passed off portions of the script to Denny for possible use in the new Exchange 2007 loader script.

UM and OCS metrics
Scripted solution to pull a count of OCS users, Voice Users, Federation enabled users, Internet enabled users and UM users and mail out the information to interested individuals. Continuing work on automating this and writing the data out with date information to a database for ongoing automated metrics.

30 Closed Magic Tickets

Exchange
43 IMAP -> Exchange 2003 Migrations
69 Exchange 2003 -> Exchange 2007 Migrations

Exchange 2007

OWA Vanderbilt Rebranding work
The login and logoff pages are complete. Premium OWA pages are about 80% done. Built test CAS and mailbox server in MS-Test environment for facilitating the rebrand testing.

Disaster Recovery Drill
Executed Disaster Recovery Drill for ITS-HCWNEM01. Successfully rebuilt and restored mailboxes.

PowerShell Scripting
Size/Quota/Message/Item # Script and Mailbox Move Script
This script is utilized to pull mailbox size, location, quota and number of items in a mailbox. This is being utilized in conjunction with a move mailbox script that records the time it took to migrate a mailbox from 2003 to 2007. This is being utilized to try and normalize mailbox move times based on size/number of items.

19 Closed Magic Tickets

Exchange
35 IMAP -> Exchange 2003 Migrations
1 VUMC -> VUExchange 2003 Migration

Exchange 2007

Migrations of ~50 mailboxes to 2007 for Pilot from the Exchange 2003 environment.

Verified\Fixed Network and System configuration on Exchange 2007 windows boxes. – This was in support of the effort to quickly put Exchange 2007 into pilot.

Got Exchange 2007 Mailbox servers configured to backup with Legato Exchange Module.

Tested/configured resource room account in Exchange 2007 in preparation to be able to utilize internal Exchange 2007 mechanics for resource accounts rather then auto-accept scripts

Extensive troubleshooting to determine issues caused by Exchange 2007 Public Folders in the environment that were brought to light by MC removal of one of their Public Folders.

Troubleshoot CAS and Hub issues.

Complete restores of internally affected mailboxes from Disaster Recovery test that caused disk corruption.

Cleanup and complete configuration of Exchange 2007, UM and UC boxes in DiamondIP.

General Windows Admin

Extensive Troubleshooting of AD issues stemming from I-Dev patching that was caught via initial outage in OCS. Including workaround to get OCS working again before the final issue was resolved. This was tracked to a DNS issue eventually.

Desktop Wallpaper setup for Servers
Updated BGInfo install procedures for allowing easier installation and lowering issues with users that were not admins on the servers.

Terminal Server License Research
Continued work in determining proper configuration of Terminal License Server for Vanderbilt’s needs.

PowerShell Scripting
Delete Files script
Utilized for keeping server logs and directories within our Windows servers clear. Options for how many days you want to keep in a directory or directory and it’s subdirectories.
Daily Exchange Metrics
Scheduled combined daily metrics created for both Exchange 2003 and Exchange 2007. This is to keep track of mailbox locations/size and total items.
Weekly Exchange Metrics
Scheduled scripts that pull # of mailboxes in VUExchange 2003 and VUExchange 2007 as well as available diskspace on each mailbox server and are mailed to the Exchange team.
Weekly Size/Quota of Specific Mailboxes
Scheduled scripts that pull quota and usage of interested mailboxes and mails out the information on a weekly basis to more accurately keep track of quota issues on mailboxes that aren’t utilized by specific users.
VUSpaceGroups ITS space
Scripted automated pull of file and disk information for assisting in cleanup of the ITS VUGroupspace.
Daily Exchange Process Monitor
Scheduled a script to daily pull all processes running on the Exchange servers and the associated paged/non-paged current and peak memory usage to determine if McAfee was the culprit in a Mail Server lockup that seemed to point to a memory issue. After running for 2-3 weeks up to patching time it was noticed that the McAfee process on the Exchange Server Backends were not releasing memory. A ticket is in process with McAfee to determine the cause.

10 Closed Magic Tickets

Exchange
22 IMAP -> Exchange 2003 Migrations
1 VUMC -> VUExchange 2003 Migration

Exchange 2003 Metrics
Database created for exchange metrics. Initial population via powershell wmi scripts in progress. Working to create automated reports for automated monthly reports

Exchange Patching
Emergency patching of the VUExchange environment, in response to critical patches being released by Microsoft, to keep our service secure.

Extensive Exchange Mailbox Restores

Loader Change
Assisted in preparing/troubleshooting the loader change to have new mailboxes created on MAILBE16. This is to better balance mailbox distribution in our environment.

Exchange 2007
Moved Sysadmin to Exchange 2007 mailboxes
Rebuilt Unified Messaging server and OCS Mediation Server into Production Exchange 2007 Environment to support Enterprise Voice in Production.

General Windows Admin
Desktop Wallpaper setup for Servers
Distributed process for updating our servers with a uniform desktop wallpaper that gives system information and the server name in an appropriate production (red), non-production (green) color. This is part of the process to insure better quality when dealing with production/non-production servers.
SCOM Box (ITS-HCWNAP08)
Installed new management packs for ISA, Exchange 2007, OCS, Print Services. Updated management packs for Exchange 2003, SQL, Sharepoint. Continued cleanup and overrides of superfluous errors and warnings. Continued work on audible alerts for the NOC for consumption of SCOM alerts.
Enterprise Terminal Server Licensing
Spent time reconciling Terminal License CAL usage with historical software store purchases.

16 Closed Magic Tickets

Mail Admin
Continue cross-training on Proofpoint appliance