Exchange 2007 Deployment

The majority of the month has been working on project documentation, hardware requirements, and investigating different archiving solutions.

One of the things that I realized needs more documentation in the messaging community is Public Folders. There really is very little guidance on sizing hardware for dedicated Public Folder servers. Here is what Microsoft has written on Scalability. I know in Exchange 2010 they are being deemphasized even more and Sharepoint is probably a better place for them, but when you have over 8,000 Public folders; I think we might have them for a little while.

One of the issues that has been plaguing the deployment has been that after Exchange Rollup-6 was deployed, Entourage clients were unable to see any messages in the inbox. The client would connect, but no messages were displayed. The rollup-6 rollout was a red herring, which coincided with a server rebuild, and mailbox migration.

After numerous hours of troubleshooting, a call to Microsoft Premier Support was made. Here is what I learned. (thanks Pawan Kapoor!)

It seems that even though all other clients do their authentication on the Client Access Server, but Entourage authenticates and uses ASP.net. on the Mailstore role This means that you will need to add the ASP.net feature along with the integrated and windows authentication features on each mailstore server you deploy.

Here are the two KB articles that the Technician sent to me.

Configuring Authentication in IIS 7.0:

You cannot connect to your mailbox on Exchange Server when you use Entourage for Mac:

Office Communication Server Deployment

The pilot is now in full swing. The adoption rate of this product is astounding.
It is really amazing to see the ease of extemporaneous collaboration that the new R2 release has achieved. Just tonight, I was sitting working on this report, when a colleague asked me to look over a document before he published the final version. I was able to see his desktop inside my Communicator client, make some word change suggestions, point the mouse to where I was referring to, and then being a second set of eyes to verify accuracy, All 131 miles apart. Through on top of that, that we were able to talk over a peer to peer VOIP connection and discuss the changes. This is truly where Unified Communications comes together.

FaxServer Replacement

The hardware named FaxSrv1 was decommissioned, and removed from the data center. The newer hardware formally known as FaxSrv2 was formatted and is currently being reloaded with Windows 2008 Standard on it. There seems to be an issue finding RAID controller drivers, this is where we currently are and looking for alternatives to this dilemma.

• Leave a comment...

Exchange 2007 Deployment

The exchange 2007 deployment has started out pretty well, we have moved all of ITS over to the new system. There have been very few issues. One of the largest issues has been with the MAC Entourage client. Once we rolled out Rollup 6, the Entourage client is no longer able to see the contents or folder structure of the mailbox. This is all Entourage clients, even the new beta one.

On the positive side, there have been numerous other clients that have been able to attach to the new Exchange 2007 Systems, including two different versions of Thunderbird, and one version of Alpine. These are all using the IMAP settings, but seem to working as expected.

The next steps are to work through the issues presented by the entourage clients, and then we will continue to retire the VUexchange 2003 systems.

Office Communications Server 2007 R2 Deployment

This has been the month for OCS, we have been anticipating the Lighthouse engagement, and the rollout of Office Communication Server 2007 R2 since last October. We have built the infrastructure, 5 physical servers, an Enterprise front end, a SQL Backend, a mediation server, and Edge server, and a Unified Messaging server. This will allow us to implement presence, IM, federation, public Internet connections, voice mail, and remote enterprise phones.

The deployment has been successful; we have normalized our phone numbers to an E164 format, and have the SIP gateway doing the conversion that the PBX is expecting. Learned more about regular expressions while doing this, regular expressions are very robust and powerful once you understand some of the syntax.

We currently have 175 people provisioned on the Office Communications Server.

• Leave a comment...

January has been one of the most productive months in a long while. Some of the projects I am on are starting to move forward. Which meeting and discussing are nice, but actually getting down to the technical work is a whole lot more fun!

Exchange 2007 Deployment

One of the first things we had to work on this month was the ramifications of adding a GC into our existing Campus AD Site. When we added the GC that was required for the prepdomain in the 2007 pre-deployment process, we didn’t realize that Exchange 2003 would pick it up in the DSaccess tab, and use it to expand groups. Since the GC is not in the same domain as the exchange servers, when it expanded the groups, it sent back errors. These errors were particularly felt when it was a security permission set on the groups.

The solution was to take the DSAccess off of automatic and manually set the GC’s that it would use for group expansion.

After that configuration was implemented and settled down, we installed three Hub transport servers into the new Exchange 2007 Administrative group. We then left the servers in place, and monitored mail flow, once we were certain that mail still flowed appropriately we then went ahead and introduced the Client Access servers (CAS) and the Mailbox servers.

We are now working on the configuration of the host based firewalls, and the load balancer configurations. After that our Pilot group of users will be ready to help iron out any irregularities’ that may have been introduced.

One of the lessons learned when deploying a new routing group connector was that each server in the legacy exchange organization that will be sending mail across the connector must be in the universal group Excahngelegacy interop group. This will grant them permission to send mail across the connector.

Octel replacement

We are currently looking at the bids again to further determine whos product would be the best fit for Vanderbilt.

Fax server deployment

This project is currently not being activity worked on. It is awaiting confirmation of the Unified Communications Project.

Unified communications project (lighthouse)

We have been accepted into Microsoft’s Lighthouse program for their unified communication product. We are currently waiting for the R2 release of Office Communications Server, and new hardware arrival. This project is dependent on the Exchange 2007 deployment project.

• Leave a comment...

AD Meetings

These meetings are still being productive. We have come up with different scenarios with the Exchange 2007 Schema Changes, and even found dependencies that we might not have seen other wise.

The big dependency is that there has to be a GC in the root domain, even if their will never be email accounts in it. This is largely due to the Universal security groups that Exchange 2007 is based upon.

Octel Replacement

No change this month; most meetings were canceled due to members not being present.

Fax server Project

This project has been put on hold. I have turned of the Faxsrv1 equipment and have started to reclaim the FaxSrv2 equipment. We will be building this out with the Windows 2008 standard server operating system.

Unified Communications Pilot

We have started using the DID lines for communications with the outside world. Some team members have seen the usefulness of having the communicator tied into the phone system. It allows them to have the head set on, that way their hands are free to troubleshoot with tech support.

The other big news is that partner support has started using the Communicator software. This has allowed partner support to be able to familiarize themselves with the product and the idiosynchrocies that come along with it. When we go live in Feburary with the full product offering they will be ready to support any issue.

Exchange 2007 design\implementation

There has been a lot of movement on the project. All the hardware has been built and ready to deploy. A total of 4 virtual machines have been built and prepped for installation.

We have procured a subnet, and created a new AD Site, and have it behind a firewall. The AD team has deployed the DC’s dedicated for this site. We are working with the security team to get the proper firewall rules in place.

The delay in this project is the schema changes and the dependency of having a GC in the root site. The decision to build a new DC and have it in the new AD site has been made to help reduce the amount of replication traffic and latency times.

• Leave a comment...

Another fine month in IT, seems the projects are slowing down a little. Well, at least the forward momentum has slowed some. Given last month’s drive to get Unified Collaboration into Vanderbilt, we know that we need to move forward, but we are taking a step back and making sure we have the proper infrastructure in place. The test environment has pretty much fleshed out what will be needed and the enormity of the undertaking, but we are up for it.

AD\Exchange 2007 meetings

This meeting has been a standing Friday meeting to help coordinate the introduction of Exchange 2007 into the current exchange organization. The mantra we are saying is “Do no harm”. We have created a plan to test the routing in our test environment, to ensure that we are not missing any last minute details that might cause “harm”. We already had our systems in the test environment so a lot of our time has been hurry up and wait. The other thing slowing this down is the availability of the medical center CMS staff, there is a lot of issues in the production environment so the test one takes a back seat.

Octel replacement

The never ending project, I keep thinking each month I will be able to say we have some to a decision, and it looks like the team has came to the another decision, and now we are working with the vendors to ensure that we have a solid Hybrid system. Now the last part of this is to finish the recommendation, and present it to upper manangement.

FaxSrv replacement

This project has been put on hold. The server that this is on, has started have a hard drive failure. It is out of warranty, and there is not any momentum on getting a fax server in place to send fax, because the unified Collaboration project is overlapping it.

Unified Communications Pilot

I have spent the most time this month on this project; it has been a lot of fact checking and configuration checking. I have also spent time on trying to get the test OCS server federated with the Production OCS server. The edge configuration has been interesting. I am seeing it trying to go out, but it is never arriving. Not sure where to go now. Validation works with a couple warnings, I have set the host file to have the hostnames for the external nics, so name resolution shouldn’t be the issue, but I am not 1005 certain that isn’t the problem.

Exchange 2007 design\implementation

The test exchange 2007 systems are working pretty well; here are still some minor details we are still working on. One being that the contacts doesn’t show up in IMAP sessions with Entourage. Have to give Kudos to Dan and Tony for figuring out what the issue was, it seems when I created the mail enabled contacts I didn’t mark one of the mail attributes.

The other outstanding issue is that of free/busy right now if you log into OWA you can see the free\busy but the Outlook client still isn’t showing it for 2003 users.

• Leave a comment...

Octel Replacement

We have had demonstrations by the top three candidates for replacing the Octel voice mail system. Each one had its good qualities, and each one had its bad. Over all each one will be a good fit with the university. It is hard with today’s technologies to find a comparable workhorse like the Octel, but for a 5 year solution they are all comparable.

Exchange 2007     

The Exchange 2007 project has come to the next phase, we are redoing the architectural diagrahm to get more users onto the system. The new architecture will be a mixture of Virtual Machines and Physical machines. This should give the best performance. We are also looking at different types of clustering that would be applicable for a 5,000 user deployment.

Office Communications Server

The requirements for this project have mandated more research on tying this implementation into the current pbx system. There were a couple of options, one is to build a Linux sip gateway, and the other would be to buy a pre-built gateway. Even though the first option gives more flexibility, the second option comes with configuration and pretested compatibility.

Proofpoint New Hardware Deployment

After purchasing new hardware last month, we have spent this month planning out how to implement the new hardware, and get the cluster upgraded to 5.03. This planning is due to the two upgrading attempts that had to be backed out. The new plan takes a cautious, more methodical approach, while getting the new hardware in place as quickly as possible.

Exchange DR

This has been one of the operation issues that have been open for most of the month. RMSE was the product that we were using, we now must upgrade to RM. This is due to a Fliar upgrade on the Clariion. This has had to be coordinated with EMC and their support, due to the different parts that have had to be upgraded. This issue is still open, while engineering works out one of their bugs.

Certifications

I passed the MCSE 2003 Upgrade Exam, Yes, that means that I am now totally certifiable. or is that certified. Now I will be looking to find a more generic Messaging certification to round out my skills.

• Leave a comment...

Certifications

I recently found out that Microsoft was going to retire their upgrade exams on March 31^st, finding this out I broke down and took the time to study for the exam to upgrade my MCSA to a current certification. Panicking thinking that I had until the 31^st to get my other exam taken, I have been cramming to prepare for my last test. When I called to schedule the exam, I was told I couldn’t get in until the 1^st of which was a day late. Noting this to the person on the phone, she informed me that Microsoft has extended the retirement date, so I have been given a reprieve, but I have renewed my MCSA from a MCSA 2000 to a MCSA 2003, I will be upgrading my MCSE in April.

Exchange 2007 Prototype

This month has been more of a refinement of issues on the Exchange 2007 environment. After putting a second person on the system, Scott Hogan, he was able to help find some of the inconsistencies that were between the systems. Then a feature of Outlook 2007 of looking for Autodiscovery in DNS when it first starts up, had a brief issue of people on 2003 getting a login prompt. This was fixed with the removal of the A record for Autodiscovery. Vanderbilt.edu. This left only the dns entry for autodiscover.uc2k7.vanderbilt.edu. From this addition we also have concluded we need to move the VM’s off the test cluster onto a more dedicated system, or physical hardware, then we can proceed to move the rest of application hosting onto the 2007 system.

Office Communications Server/Office Communicator

With the start of the prototype for Office Communicator the server side was finally stable enough to get the whole department onto it. This allowed for instant messaging to stay within our network. Some of the features that will be useful is the tying it into outlook to set free busy, and being able to do PC to PC voice communications. Some of the issues that have been noted, and are configuration changes needed, is the ability to change passwords, the ability to have a preset group to pre-populate the contacts, transfer of files, and sending hyperlinks. This is still in its early stages of development, but shows a lot of promise of being a useful tool.

Tying this into the current VOIP system will take some concerted effort.

Proofpoint renewal

This month we spent a considerable amount of time discussion with Proofpoint representatives the different offerings that were available to the university. After all was said and done, we were able to save the university a considerable amount of money, improve the service by adding Dynamic Reputation Service, and refreshing the hardware to accommodate a large growth in traffic .

Octel replacement

The project is in to the phase of narrowing it down the candidates, and scheduling demos of their proposed solutions. After we see the demos we can better judge wich system will be the right fit for Vanderbilt.

Evaluation time

March also brought in with it the need to finish up team evaluations. This was made easier since we did a six month evaluation, and let people know where improvement was needed. This also made an improvement in the overall scores. A noticeable effort was seen from the members of the teams to take the necessary steps to improve their evaluation scores, which in the long run has made a better, more dynamic team.

• Leave a comment...

System Center Operations Manager

There wasn’t much work done on SCOM during this month. The focus has been on Nagios and getting information into it. The major issue with SCOM at the moment is to make sound when an error occurs. Kendra has found powershell commands to make the sounds, now we just have to make the event trigger.

OCTEL Replacement Project

This project is now in the RFP Stage, the RFP was sent out to 5 vendors, three have responded, one has taken themselves out of the running. Now we are working on the matrix to do an apples to apples evaluation, and inviting all three that responded in to demonstrate their product. We are still on track to have this project completed by July 31^st.

Unified Communications Pilot

The exchange 2007 portion of this pilot, has been an interesting bag of worms. Getting email up and running was fairly simple. Found out that it was simpler to get a valid certificate for Active synch then to try and put the certificate on the phones. Once I placed the geotrust certificate on the server, I was able to synch up my phone. The next big issue was the free-busy information, I loaded the IOrepl software and configured exchange 2007 to use public folders for Free busy. The IORepl software showed that it was loading successful but nothing showed up. Opened a MS Support ticket, and there was a powershell command that had to be ran, that allowed different domains to load free busy information in the public folder, Once this was ran, everything was good to go.

MCSE Training schedule:

Had to change up my schedule. Found out that MS70-292 and MS70-296 was at end of life March 31^st. Now I am cramming to pass both tests before that date. The biggest issue is finding time to study.

My Medical Training

Found out this month that Doctors could call your liver names. They called mine “Fatty” seems one can get Nonalcoholic steatohepatitis
from not drinking to much. But now I have a liver that tells me when I eat something that is high in fat. So I have started working on my low fat diet that the doctor has put me one. The only good thing about this is that I can honestly say that Vanderbilt Medical Center was thourough and professional, and not only am I an employee but now have been a patient too.

• Leave a comment...

• Leave a comment...

TechEd 2007 “Make your mark in IT”

This was probably the toughest week of the month. Too much food. Too many sodas, and way to much information! I can no longer use my excuse for not learning Linux, the excuse of not being able to type; Windows Powershell is being positioned as the administrative preference for Server 2008. Maybe this is the time to transition into a more administration type roll.

One of the main things that I took away from Teched, is that IT as a whole is changing, and changing quickly, this means that IT departments need to be more “fluid”, able to adapt dynamically. Even though this was one of the “buzzwords” there is a lot of truth in it. IT departments need to look at “Service organized Architecture”. If you’re not able to provide a service to your customer, quickly, efficiently and on time, you might find your department outsourced to someone who can. The keynote, repetitively hit on the subject of “fewer administrators, more uptime, and consistent services” all being driven by the architecture.

The other interesting thing I picked up, was that The university is facing the same obstacles as other Higher education institutions in the US. We are seeing a higher demand for infrastructure from our younger students, pushing our speed from conception to deployment a whole lot faster, the days of the university being 5-7 years behind Corporate America is rapidly coming to an end.

Service Center Operations Manager

I worked with Microsoft on issues with Systems staying in a not Monitored state. Even though we worked through many scenarios and configuration changes, it all came down to the machines DNS host name was still XXXXX.ds.vanderbilt.edu, and not XXXXX.its.vanderbilt.edu. This single issue has probably taken over 40 hours to research and fix. Once again advocating the use of DS.vanderbilt.edu as an actual ADDNS, this would have by-passed the need to manually edit the DNSHostname in Active directory. This current issue has been resolved.

I worked with Microsoft on the issue of Virtual resources not being monitored on Clustered servers. The issue is that the actual hardware and OS are being monitored, but the virtual resource or application is not. We turned on Kerberos logging, and have seen Kerberos errors in the Event logs. Current Status is MS has suggested we enable Kerberos Authentication in the Cluster Manager, to allow authenticated service to the Operations Manager server. Current testing plan is to turn it on, to see if it starts the monitoring on the Operations Manager server, then turn it back off.

Email oneness.

Have been working with Steve, and Scott Hogan, to document the service gap between the Medical center offering and the University offering. This gap, even though previously know, once documented looked more formable on paper. Bridging the gaps, in a near seamless fashion, could take 12-18 months. Once presented to the Core team, concerns of speed of transition, and cost, were realized as true valid concerns for the project. Moving from an analyze phase to a conceptual design Phase. We are now doing a conceptual design for a forklift type event.

Proofpoint Load balancing.

This project started to take form this month. We met, and have fleshed out the action items that are needed to produce a test environment of clustering the proofpoint behind the CSM modules. Even though this is a new project, the team is moving forward quickly, because this will help elevate the “max children errors” being seen on Mailgate02.

Thought: He who cannot change the very fabric of his thought will never be able to change reality, and will never, therefore, make any progress.” – Anwar sadat

• Leave a comment...