Category Archives: Microsoft

Messaging and Collaboration, Sharepoint.. etc

Move Request on Exchange 2019 (During failover) will warn you that it postponed due the move of DB

The Issue:

So while working on a new Exchange Migration project, I have encountered a weird issue where I could see users migration batch status complaining about being stalled due to (Big Funnel).

The error is showing as in the below screenshot and it doesn’t occur instantly after you start the migration of the user but right after it starts.

StalledDueToTarget_BigFunnel 68.47 MB (71,795,512 bytes) 20

User StalledDueToTarget_BigFunnel 37.2 MB (39,003,538 bytes) 20

User2 StalledDueToTarget_BigFunnel 14.71 MB (15,421,154 bytes) 20

User3 StalledDueToTarget_BigFunnel 44.2 MB (46,345,009 bytes) 20

User4 StalledDueToTarget_BigFunnel 4.647 MB (4,872,404 bytes) 20

User5 StalledDueToTarget_BigFunnel 14.47 MB (15,169,768 bytes) 20

User6  StalledDueToTarget_BigFunnel 171 MB (179,280,335 bytes) 20

User7 StalledDueToTarget_BigFunnel 753.4 MB (789,980,880 bytes) 20

User8 StalledDueToTarget_BigFunnel 18.35 MB (19,236,680 bytes) 20

User9 StalledDueToTarget_BigFunnel 205.9 MB (215,951,208 bytes) 20

User10 StalledDueToTarget_BigFunnel 166.2 MB (174,243,238 bytes) 20

User11 StalledDueToTarget_BigFunnel 13.81 MB (14,481,739 bytes) 20

User12 StalledDueToTarget_BigFunnel

image

Error Message

Request ‘domain.com/CompanyUSER/Region1/User1’ (b5dbf3ff-21a1-4ec1-a29c-15b794a17386) failed.

Error code: -2146233088

Connection to the Content Transformation Service has failed.

Context:

——–

Operation: IMapiFxProxy.ProcessRequest

OpCode: TransferBuffer

DataLength: 31680

——–

Operation: IMapiFxProxy.ProcessRequest

Operation: IMapiFxProxy.ProcessRequest

OperationSide: Target

b5dbf3ff-21a1-4ec1-a29c-15b794a17386 (Primary)

OpCode: TransferBuffer

DataLength: 31680

——–

Operation: IMailbox.ExportMessages

Operation: IMailbox.ExportMessages

OperationSide: Source

b5dbf3ff-21a1-4ec1-a29c-15b794a17386 (Primary)

Flags: SkipItemValidation

PropTags: (null)

——–

>>>> Scheduled WorkItems: EnumerateFolderMessages(P:29792,R:0,S:0,C:14); EnumerateFolderMessages(P:29807,R:0,S:0,C:24,Cnt=3); WriteFolderMessages(P:0,R:0,S:0,C:686); EnumerateFolderMessages(P:30554,R:0,S:2,C:55); EnumerateFolderMessages(P:30612,R:0,S:0,C:36,Cnt=2); WriteFolderMessages(P:3,R:0,S:0,C:301); EnumerateFolderMessages(P:30975,R:0,S:1,C:21); WriteFolderMessages(P:2,R:0,S:0,C:97); EnumerateFolderMessages(P:31094,R:0,S:0,C:18,Cnt=6); EnumerateFolderMessages(P:31279,R:0,S:0,C:19)

————–

The Microsoft Exchange Mailbox Replication service was unable to save changes to request.

Request: ‘9a444721-80e2-4cf8-8c81-8a3afe3dc775’ (bbc2c66e-857e-4ba6-8462-9d66da73d400)

Database: DB01

Error:

The request has been temporarily postponed because a database has failed over. The Microsoft Exchange Mailbox Replication service will attempt to continue processing the request when capacity becomes available on the new server hosting the database.

image

Looking at the event ID number 1114 it mentions there seems to be an issue with the request seems there might be an issue with the mailbox being moved.

To dig deeper I am going to search some of the users reporting the same error by using their GUID

image

The property “DisplayName” with value “User LastName” is invalid. The value can’t contain leading or trailing whitespace.

Solution: (For a single user)

To resolve the problem, I am going to remove the trailing space in the end of the display name. You can safely use the below Powershell script to solve this problem however, if you don’t trust yourself or you’re not familiar much with Powershell, You can try it on a lab or a single test user for instance.

Get-Mailbox -Identity USER | Foreach { Set-Mailbox -Identity $_.Identity -DisplayName $_.DisplayName.Trim() }

image

Solution: (For all users)

Get-Mailbox | Foreach { Set-Mailbox -Identity $_.Identity -DisplayName $_.DisplayName.Trim() }

clip_image001

Some relevant errors you might encounter as you’re moving users to Exchange 2019

Error code: -2146233088

Connection to the Content Transformation Service has failed.

Context:

——–

Operation: IMapiFxProxy.ProcessRequest

OpCode: TransferBuffer

DataLength: 31680

——–

Operation: IMapiFxProxy.ProcessRequest

Operation: IMapiFxProxy.ProcessRequest

OperationSide: Target

eecb073e-e694-4bbc-8652-54dc05a351ea (Primary)

OpCode: TransferBuffer

DataLength: 31680

——–

Operation: IMailbox.ExportMessages

Operation: IMailbox.ExportMessages

OperationSide: Source

eecb073e-e694-4bbc-8652-54dc05a351ea (Primary)

Flags: SkipItemValidation

PropTags: (null)

——–

>>>> Scheduled WorkItems: EnumerateFolderMessages(P:14014,R:0,S:0,C:13); EnumerateFolderMessages(P:14029,R:0,S:0,C:15,Cnt=2); WriteFolderMessages(P:1,R:0,S:0,C:132); EnumerateFolderMessages(P:14192,R:0,S:0,C:17); WriteFolderMessages(P:1,R:0,S:0,C:48); EnumerateFolderMessages(P:14259,R:0,S:0,C:12,Cnt=4); EnumerateFolderMessages(P:14320,R:0,S:1,C:15); EnumerateFolderMessages(P:14337,R:0,S:0,C:20); WriteFolderMessages(P:2,R:0,S:0,C:126); EnumerateFolderMessages(P:14485,R:0,S:0,C:30)

Search and Delete certain Items/Folders from a Mailbox

The Story

During a project of Hybrid migration from Exchange on-premises to Exchange online, I was almost about to finalize the project by moving the last remaining users mailboxes however had an interesting issue to deal with where a user was failing with the following error:

The Error after migration:

Error: MigrationPermanentException: Mailbox dumpster size 50.87 GB (54,620,074,576 bytes) exceeds target quota 30 GB –> Mailbox dumpster size 50.87 GB exceeds target quota.

image

After some research it turned out that you can clean the dumpster using search-mailbox PowerShell cmdlet, Sync the user’s object with ADConnect and then continue the migration from the last failure.

To solve the issue, Go on your Exchange on-premises and launch Exchange Management shell

Solution applied:

First, Let’s see the user’s dumpster and recoverable items

Get-MailboxFolderStatistics -Identity “User” -FolderScope RecoverableItems | Format-Table Name,FolderPath,ItemsInFolder,FolderAndSubfolderSize

image

To Delete the dumpster only use this

Delete dumpster only

Search-mailbox -identity User -SearchDumpsterOnly –DeleteContent

To delete a certain email with certain subject in the dumpster use the following:

Get-mailbox “user”| search-mailbox –searchquery “Subject:’*'” –DeleteContent –SearchDumpsterOnly

image

The cmdlet will search and delete

clip_image001

image

Reference:

https://docs.microsoft.com/en-us/microsoft-365/compliance/search-for-and-delete-messagesadmin-help

Deleting Old Skype for Business or Lync server from ADSI

The story

I had a project few weeks ago where my client wanted to install Skype for Business 2019 but had installed Lync before and removed the server without doing proper decommissioning which kept dirty records in AD database and had to be removed manually in order to make a new clean installation of Skype for Business 2019

To do so:

There are two days of doing so, One is using ADSIEdit and ADUC to remove Computer Objects and Users related attributes and Security Groups.

I normally would prefer PowerShell but since we can demonstrate both ways for people who like to work with GUI

Starting with GUI

Removing Legacy Lync server from the AD Schema

Prerequisites

  1. Using a domain or enterprise admin
  2. Access to the ADSIEdit.

Goal of removing Legacy Lync server from your AD environment.

  1. Preparing AD schema and domain for a new deployment after you improperly deleted Lync Servers without uninstalling them.
  2. Cleaning Users’ Lync related attributes for the new deployment.

clip_image001

clip_image002

Step#1: Remove permissions

This step removes the original Lync permissions from the active director.

  1. Open Active Directory Users and Computers
  2. Right click on your top level domain being cleaned and select Properties
  3. From the Properties windows, select the Security tab.
  4. Remove all security users titled RTC*
    These are usually
    – RTCUniversalServerReadOnlyGroup
    – RTCUniversalUserReadOnlyGroup
    – RTCUniversalUniversalServices
    – RTCUniversalUserAdmins

From <http://blog.armgasys.com/?p=320>

clip_image003

clip_image004

  1. Repeat the same steps for each of the following AD Folders and

    OUs
    NOTE: Not all RTC permissions will exist in each AD Folder or OU, but these three OUs do:
    – Domain Controllers
    – System
    – Users

Domain Controllers

clip_image005

Systems

clip_image006

Users

clip_image007

Step#3: Additional AD cleanup

  1. Open Active Directory Users and Computers
  2. Drill down as follows
    [Your Domain] \ Program Data \ Distributed \ KeyMan
  3. Delete LyncCertificates
    NOTE: This may not exist in all scenarios.
  4. Drill down as follows
    [Your Domain] Users
  5. Delete all RTC* and CS* users created by Lync
    I.E. CSAdministrator, CSHelpDesk, RTCComponentUniversalServices, Etc.

image

Deleting users from the User OU

clip_image001[6]

Deleting CS Users

clip_image002[4]

Step#4: Cleanup existing users

This steps resets Lync attributes for any domain users and contacts.

image

The Second way: Using PowerShell

get-aduser -filter {msRTCSIP-PrimaryUserAddress -like “*”}|set-aduser -clear msRTCSIP-PrimaryUserAddress,msRTCSIP-PrimaryHomeServer,msRTCSIP-UserEnabled,msRTCSIP-OptionFlags,msRTCSIP-UserPolicies, msRTCSIP-DeploymentLocator, msRTCSIP-FederationEnabled, msRTCSIP-InternetAccessEnabled

Result:

Users attribute are clean and AD has nothing left over of Previous installation of Lync or Skype for Business .

clip_image001[8]

In Exchange MRSPROXY.SVC FAILED BECAUSE NO SERVICE WAS LISTENING ON THE SPECIFIED ENDPOINT.

Symptoms

In Exchange MRSPROXY.SVC FAILED BECAUSE NO SERVICE WAS LISTENING ON THE SPECIFIED ENDPOINT. THE REMOTE SERVER RETURNED AN ERROR: (404) NOT FOUND

Exchange 2010 / 2013

You get an error when you’re trying to setup Hybrid configuration between your Exchange On-premises or Online.

After I had one issue like this I did some research and used Fiddler / Wireshark to check for traffic I noticed that the traffic on the server is not encrypted and testing the Migration Server Availability was reporting that the MRS service was not listening on the supposed port which is 443.

CAUSE


This problem may occur if the ExchangeGUID property of the Exchange Online MailUser object does not match the ExchangeGUID property of the on-premises mailbox. To successfully move a mailbox, the value of the ExchangeGUID property in the Exchange Online mailbox and in the associated on-premises remote mailbox must match.

image

In this case the solution was pretty easy, but still you’ll have to make a hard choice of choosing to place Exchange behind a load balancing with SSL Offloading on or not.

In my case I had to turn off the SSL Offloading on the Load balancer and that alone was enough to get this working.

Resolution:

Make sure that SSL Offloading is disabled on OWA/OA and Load balancer if there’s one.

Other resolutions:

https://support.microsoft.com/en-us/kb/3065754

Powershell script to audit users who authenticated against DC servers

The story:

I have got a request from a client asking to find out which server(s) is using which domain admin or a highly privileged account as a service.

To find this I already wrote a powershell script that does this, Search the non standard/(Domain only users) and show the services and name of the servers where those accounts are configured on utilizing Remote powershell to do so and the use of a Domain Admin user.

You can refer to this link to see this article by clicking here

Creating the script process:

The same client wanted to also know which of those accounts did authenticate and wanted to know from which server/Computer did the request originate from and to which DC did it go.

I have started thinking of the process of doing so by again utilizing remote PowerShell to check against certain security events on AD to check which user among the Domain admin members did authenticate.

After sometime and with the help of some forums I managed to get script ready which looks in all Domain Controllers for users that are members of the Domain Admin groups who triggered an event ID 4624 and from which Computer did this request came from.

The Script :

# Get domain admin user list
$DomainAdminList = Get-ADGroupMember -Identity 'Domain Admins'
# Get all Domain Controller names
$DomainControllers = Get-ADDomainController -Filter * | Sort-Object HostName
# EventID
$EventID = '4624'
#
# Get only last 24hrs
$Date = (Get-Date).AddDays(-3)
# Limit log event search for testing as this will take a LONG time on most domains
# For normal running, this will have to be set to zero
$MaxEvent = 100

# Loop through Dcs
$DALogEvents = $DomainControllers | ForEach-Object {
    $CurDC = $_.HostName
    Write-Host "`nSearching $CurDC logs..."
    Get-WinEvent  -ComputerName $CurDC -FilterHashtable @{Logname='Security';ID=$EventID;StartTime = $Date} -MaxEvents $MaxEvent |`
    Where-Object { $_.Properties[5].Value -in $DomainAdminList.SamAccountName } |`
    ForEach-Object {
        [pscustomobject]@{SourceIP = $_.Properties[18].Value; SamAccountName = $_.Properties[5].Value;Time = $_.TimeCreated;LogonEventLocation = $CurDC}
    }
}
$DALogEvents

How to run:

The Script must be run on DC with a privileged account in order to get the write results, The default time interval is set to 3 days but you can choose to increase that.

You can also change the default group where you want to search for members by changing Domain Admin groups to something else.

Screenshot of the result

Slow Migration – Office 365

The story:

In office 365 when you’re working on Exchange 2010,2013, 2016 or 2019 in a hybrid environment things might look easy but in a big enterprises where Internet security is something being taken into account very seriously. It might cause many issues that you don’t expect at all.

One of my clients whom I was doing Exchange Migration for had an issue with the Migration. The error was as follows:

Error occurs after Office 365 Exchange online connects to Exchange on-premises 2010 mailbox server

Error in Office 365

         : 20.

                                           27.04.2016 08:03:17 [DB3PR05MB0778] Transient error DataExportTransientExcep

                                           tion has occurred. The system will retry (2/1280).

                                           27.04.2016 08:04:53 [DB3PR05MB0778] The Microsoft Exchange Mailbox Replicati

                                           on service ‘DB3PR05MB0778.eurprd05.prod.outlook.com’ (15.1.466.25 caps:03FFF

                                           F) is examining the request.

                                           27.04.2016 08:04:55 [DB3PR05MB0778] Connected to target mailbox ‘lcwonline.o

                                           nmicrosoft.com\ec96e315-1059-4710-b358-1c4b42f3edeb (Primary)’, database ‘EU

                                           RPR05DG049-db131′, Mailbox server ‘DB3PR05MB0778.eurprd05.prod.outlook.com’

                                           Version 15.1 (Build 466.0).RequestExpiryTimestamp                   : 03.04.2116 07:42:38

ObjectState                              : New

Troubleshooting:

To troubleshoot issues, You need to put so many things into account! The architecture of the infrastructure of where you are doing the project is very important and the need of knowing how things are working matters.

Things that could always come in mind and handy are what you will need to start your troubleshooting:

– Bandwidth Limitations or Performance:

https://technet.microsoft.com/en-us/library/dn592150(v=exchg.150).aspx

https://technet.microsoft.com/en-us/library/jj200581(v=exchg.150).aspx

– Exchange Configuration (MRS)

To troubleshoot the MRs, You need to know what kind of error you’re getting and to see this you can use the following powershell after you connect to Office 365 powershell.

Get-MoveRequest {email} | Get-MoveRequestStatistics -Diagnostic -IncludeReport | Export-Clixml c:\logfile.xml

The resultant report will reveal the error and shows you where is the exact culprit.

– Disk Latency
– Firewall Configuration (IPS/IDS)

From Exchange 2016 to 2019 or 2013 to 2016 The transient error might be related to MRSProxy or at least this is the case with me 90% of the time. To resolve this issue you will need to change the MRSProxy values on the target server and depending on the error might also be the Source server too.

SOLUTION:

===========

1. Some instability was detected in communications as well as saturation by the size of the link.
2. The procedure to increase the timeout for the service through the file MRSProxy

File: MsExchangeMailboxReplication.exe.config

Object / line: DataImportTimeout.

New Value: 00:10:00

clip_image001[4]

New Configuration

clip_image001[6]

HOW TO GET ALL DOMAIN JOINED SERVER SERVICES THAT USING A UNIQUE OR DOMAIN USER

The Story (Finding Domain Joined Servers Services users)

If you’re wondering which of your servers are using domain joined account or a non regular account like network service or system. You will need to go through every server’s service console and check that one by one but thanks to PowerShell this job was made like a piece of cake.

Requirement

The requirement to run this script is a domain admin account since the PowerShell will require access to other servers using Remote PowerShell using Invoke command and run a Get-WMIObject script to find out those details. So in short I will write the required things for this to work

1- Logged in to Active Directory (In order for AD PowerShell module to run and find computers).

2- Domain admin account (To run the remote PowerShell on other servers and get service details)

3- Firewall for domain joined computers is open (To allow remote PowerShell to work) or have remote PowerShell enabled via GPO.

The Script will also show you the offline (inaccessible servers) and will state those servers as down as you can see in the screenshot below.

The script will also prompt you for a path to save the output. You can enter something like C:\Services.csv  as soon as you type the file path and extension it’ll be opened using Notepad.

image

#Check servers down and get services from the responsive servers

$Computers = Get-ADComputer -Filter { OperatingSystem -Like ‘*Windows Server*’}

$Input = ForEach ($computer in $computers){

$comp = $Computer.DNSHostName

$dist = $Computer.DistinguishedName

if (Test-Connection -Computername $comp -count 2 -Quiet )

{

Invoke-Command -ComputerName $comp -ScriptBlock {Get-WmiObject win32_service | where {$_.StartName -notlike “*LocalSystem*” -and $_.StartName -notlike “*LocalService*” -and $_.StartName -notlike “*NetworkService*” -and $_.StartName -notlike “*System*”} | select DisplayName,StartName,State }}

else{ Write-host $comp is down -foregroundColor red -BackGroundColor black

}

}

$Output = Read-Host “Enter File path and Name to save output to”

Out-File -FilePath $Output -InputObject $Input -Encoding ascii

Notepad $Output


image

image

Lync Front end event ID 32178

The Issue: Replication problem occurs between Frontend Servers and Edge

If you look at the Eventviewer you might find the following error when you have finished deploying Edge or Frontend Server.

Error:

Failed to sync data for Routing group {563F57A3-0AC1-560A-91B4-74ACDECB2683} from backup store.

Cause: This may indicate a problem with connectivity to backup database or some unknown product issue.

Resolution:

Ensure that connectivity to backup database is proper. If the error persists, please contact product support with server traces.

clip_image001

This error happens when you mistakenly import a certificate that’s not self-signed into the Local computer’s trusted authority store.

To fix this issue you will have to use the following powershell cmdlet on all the servers that are showing the error
  • Get-Childitem cert:\LocalMachine\root -Recurse | Where-Object {$_.Issuer -ne $_.Subject} | Format-List * | Out-File “c:\computer_filtered.txt”
  • Then type the command
  • Notepad C:\computer_filtered.txt
  • The command will open notepad file with the certificates details in it.
  • Take a note of all certificate’s thumb print number and open your MMC console
  • Click File -> add the Certificate then choose Local computer
  • Navigate to “Trusted Root Certification Authority” store

clip_image002

Check the certificate that their thumb print were shown in the Txt file and remove to the Intermediate Certification authority store.

If you have many certificates in the Trusted root store, you can manage the view and choose “Issued by” and then click on the certificate that the “Issued to” and “Issued by” do not match

clip_image003

double click on it then choose the thumbprint section and try to see if this thumbprint value matches the one in the text and move it to the intermediate store.

clip_image004

When you finish, you must restart the servers one by one in order to resolve this issue and then you will notice that the error is gone and that services are back to normal state

clip_image005

http://support.microsoft.com/kb/2795828

Lync common issues

Publishing Topology on Frontend

Some of the issues that might face you while you’re deploying Skype for Business/Lync is something that might be stuck within AD.

A previous installation that was not properly cleaned could result in a failure of publishing the Topology.

Here is one issue that I had while deploying one for one of my clients.

Issue

Enable-CsTopology : Multiple Active Directory entries were found for type “ms-RTC-SIP-EdgeProxy” with ID in a multiple Domain Environment

Active Directory Issue?

Enable-CsTopology : Multiple Active Directory entries were found for type “ms-RTC-SIP-EdgeProxy” with ID

“lyncedge.domain.local”.

At line:1 char:1

+ Enable-CsTopology

+ ~~~~~~~~~~~~~~~~~

+ CategoryInfo : InvalidData: (:SourceCollection) [Enable-CsTopology], InvalidDataException

+ FullyQualifiedErrorId : DuplicateADEntry,Microsoft.Rtc.Management.Deployment.ActivateTopologyCmdlet

clip_image001

clip_image002

clip_image003

Solution:
  • Open ADSIEDIT and look in the following snapshot. Open Configuration for your DC
  • Collapse the menu and click on Services
  • Click on RTC Service
  • Click on Global Settings and on the right pane look if there’s any duplicated entries and remove them.
  • As you can see on my right pane I have 2 duplicated (msRTCSIP-EdgeProxy) and I’m going to remove one of them and see if I can publish my topology or not. But before that I will have to make sure that I export the entry that I wanna delete.

clip_image004

I right clicked on the last value and deleted it and here how it became now.

clip_image005

Now I will try to publish my topology and see what happens, my topology publishing failed with a new error this time.

clip_image006

I will have to go and check where’s this coming from, since it mentions TrustedService. I will go look in the trusted service

This is not going to be easy, as you need to becareful where you look .. You will need to make sure that you’re looking at the right FQDN

clip_image007

Here I could find the value MRAS for the FQDN Edge server

So I looked here and found 2 identical entries with a different (CN) if you scroll down you will see that the GruuId is the same, FQDN is the same, port is the same.

clip_image008

clip_image009

Let’s delete one of them and see again if we can publish our topology, So I deleted the one that starts with {b344}

I will do this using the Lync Powershell, you can see below that the Topology was published successfully.

clip_image010

To resolve the warning you will have to issue the cmdlet Enable-CsAdForest after the Enable-CsTopology

clip_image011

2- SKYPE FOR BUSINESS EDGE SERVER DEPLOYMENT AND HYBRID INTEGRATION WITH SKYPE FOR BUSINESS ONLINE

Skype for Business Edge server deployment and Hybrid integration with Skype for Business Online

In the last Skype for Business post I have upgraded my Lync 2013 to Skype for Business (Click here to go to that post). in this article I am going to install Edge server for Skype for Business to the same Lync Environment where I have done the Upgrade to Skype for Business.

Configuring Edge Server

Setup NETBIOS

In order to configure Skype 4 Business Edge, we’ll have to change the Netbios to give it the name of our Domain but we won’t join it to the domain.

clip_image001[8]

clip_image002[8]

Setup NICs

Edge Server must have 2 NICs, one Local NIC will point out to the Front end server but must not have Default gateway so traffic can only flow through the DMZ out to the internet and back in. but still it must be able to ping to the FE from Edge and vice versa.

The DMZ network could have a single DMZ address (Public Address to be pointing to) or three DMZ addresses for public IP addresses with standard https ports.

clip_image003[8]

clip_image004[8]

Configure Hostnames

Edit the Edge server’s host file to include Lync FE and DC’s IP addresses and Hostname

image

Install Prerequisites

  • Microsoft .Net Framework 3.5
clip_image001[10]

Now I will go back to Skype for Business FE server, I’ll launch the topology builder and add new Edge server

I will add the first Edge pool which contains of a single Edge server

clip_image002[10]

Next, you will have to choose if you want to enable federation with partners or other service providers …e.g. (Google)

clip_image003[10]

I am intending to use a single Public IP address with a different ports (nonstandard) since this is a lab. For production use it’s recommended to have 3 public IP addresses, One is for Access Edge, AV and WebConf services.

clip_image004[10]

Next I will choose the last option which says that the Edge pool is translated by NAT. I will configure my firewall to NAT ports to the Edge’s DMZ IP addresses from the Public so I am choosing this option.

clip_image005[8]

clip_image006[8]

This is the FQDN’s the default configuration .. It’ll only use a single FQDN for all services if you’re going to use a single public IP address with a different ports.

IMPORTANT NOTE

When you use a single IP address with a different ports, the Access Edge port will normally change to 5061 (Not 443 like in the _sip._tls.domain.com) SRV record which will cause failure if you forgot to change this port to match the one in your Topology’s Access Edge settings.

Next I’ll have to enter my Edge server’s Local IP address.

clip_image007[8]

Next I will be asked to enter the DMZ’s IP address which the wizard calls (Private External IP address)

clip_image001[12]

Here I am going to place the NAT IP address which is my Public IP address.

clip_image002[12]

Next I’ll have to choose which Lync FE pool will be used as the next hop to the Edge pool. In this case I’ll be choosing my main pool since the second is only for resilience purpose.

clip_image003[12]

Then I’ll associate the mediation pool for Edge server for external media traffic. I can assign both in this case.

clip_image004[12]

Now I’ll click on Finish and right click on the Site name’s properties to enable the SIP federation and XMPP federation then Publish the topology.

clip_image005[10]

clip_image006[10]

clip_image007[10]

clip_image008[10]

Now I will setup Azure Active Directory Sync on my DC server in order to sync the required users for the test purpose.

My domain is adeo.local so I want to change the UPN for users to match the synced domain. (Adeo-office365.ga) and moh10ly.com

clip_image009[8]

Installing Azure Active Directory Sync

Now I will install the prerequisites which consist of the following

clip_image010[8]

Net framework 4.5.2 is required for AADS but it’s already installed on my server

clip_image011[8]

Next I will install Microsoft Online Service Sign in assistant

clip_image012[8]

Next I will install Azure AD Module

clip_image001[14]

Finally Azure AD Sync

clip_image002[14]

Before moving forward, I’ll have to go to the Office 365 portal and activate DirSync

clip_image003[14]

Then use a global admin credentials from O365.

clip_image004[14]

Adding the forest using an enterprise admin user account

clip_image005[12]

clip_image006[12]

Due to the fact that my domain adeo-office365.ga’s public dns host doesn’t have SRV configuration because it’s hosted by the famous free domain service (Freenom) so I’ll have to add my original domain moh10ly.com as Lync (S4B) requires SRV records to point to the on-premises lync.

clip_image007[12]

clip_image008[12]

clip_image001[16]

clip_image002[16]

clip_image003[16]

I will only sync one OU, so I will untick the Sync now box and click on Finish

clip_image004[16]

I will go to the following path

“C:\Program Files\Microsoft Azure AD Sync\UIShell” and create a shortcut for the GUI application of AADS on the desktop

“C:\Program Files\Microsoft Azure AD Sync\UIShell\miisclient.exe”

clip_image005[14]

To get this GUI app to work, you will have to sign out of your account and sign back in as your username will be added to the local administrators and have the authority to open it

Log off, log back in

clip_image006[14]

Next I will go to the connectors tab and double click on the ADDS connector (Adeo.local)

clip_image007[14]

I will go to the Configure Directory Partitions and under Credentials I’ll choose “Alternate credentials for this directory partition” then enter my on-premises AD Enterprise admin credentials

clip_image008[14]

I’ll click on Containers

clip_image001[18]

I’ll untick the DC=Adeo,Dc=Local box and only choose Dirsync OU then click OK and apply

clip_image002[18]

Before I start syncing my AD , I will go to Skype for Business Server and add my domain moh10ly.com as a SIP domain

clip_image003[18]

Next I am going to change the FQDN of the SIP access edge for public domain to moh10ly.com and the default port for the Access Edge to 443 and publish the topology

clip_image004[18]

clip_image005[16]

I needed to finally check if all my FE servers are replicating. So then I can move to Edge server to install Lync components

clip_image006[16]

On the Edge server, I’ll use ISO for Skype 4 business to install the setup

clip_image007[16]

clip_image008[16]

First thing I’ll install the local Configuration Store

I’ll click on Run and then I’ll be asked to import the configuration file which I’ll must export from Lync FE (Skype 4 b FE) server

clip_image009[10]

In this case, I’ll go to Lync FE and open Lync Management shell and enter the following Cmdlet

Export-CsConfiguration -FileName c:\top.zip

clip_image010[10]

This cmdlet will export a file to the root C drive . I’ll copy this file to the edge server.

clip_image011[10]

I’ll click next to continue, this should start installing the local store

clip_image012[10]

clip_image013[8]

clip_image014[8]

Next I’ll request a certificate for Internal NIC For edge server

clip_image015[8]

clip_image016[8]

clip_image017[8]

Configure Certificate

I’ll take the CSR (Certificate sign request) code and get a certificate from my local CA

clip_image018[8]

I’ll open MMC and add Certificates console and import the PKCS certificate

clip_image001[20]

clip_image002[20]

After importing the certificate I’ll assign it to the internal NIC by clicking on Assign to the Edge Internal

clip_image003[20]

clip_image004[20]

clip_image005[18]

clip_image006[18]

Once we assign the certfiicate to the internal edge. The replication service for Edge and FE will start working

clip_image007[18]

Now I’ll import my Public Certificate to Edge Server’s DMZ NIC

I already imported my public certificate, now I’ll go to the S4B wizard and assign it there

clip_image008[18]

clip_image009[12]

Unlike IN lync 2013 when you Click on Start service in the Wizard all services start on their own but on Skype for business you ‘ll have to start the services manually by yourself.

clip_image010[12]

So Instead I used the service console to start the services.

Now I’ll go back to the FE And enable remote connectivity to Skype for Business from outside and make sure that replication works fine by checking the Topology or from cmdlet

clip_image011[12]

clip_image012[12]


After the replication is finished, I was able to login remotely with my Skype for Business on-premises accounts.

Setting up Hybrid integration with Skype online for Business (O365)

https://technet.microsoft.com/en-us/library/jj205126.aspx

https://technet.microsoft.com/en-us/library/jj204669.aspx

In order to allow Hybrid environment to function properly, we’ll have to federate our Skype for Business on-premises’s Edge server as Microsoft says below

Federation allows users in your on-premises deployment to communicate with Office 365 users in your organization. To configure federation, run the following cmdlets in the Skype for Business Server Management Shell:

From <https://technet.microsoft.com/en-us/library/jj205126.aspx>

On the front end server, we’ll run the following CMDlet

Set-CSAccessEdgeConfiguration -AllowOutsideUsers 1 -AllowFederatedUsers 1 -UseDnsSrvRouting -EnablePartnerDiscovery $true

clip_image014[10]

Next cmdlet will create a new public federated provider for skype for business online.. However it already exists by default as in the below snapshot but just to avoid any issues I will delete the default provider from control panel and recreate it again. 

clip_image015[10]

I’ll delete the hosted provider “Skype for Business Online”

clip_image016[10]

I’ll try the cmdlet again after deleting the provider ..

New-CSHostingProvider -Identity SkypeforBusinessOnline -ProxyFqdn “sipfed.online.lync.com” -Enabled $true -EnabledSharedAddressSpace $true -HostsOCSUsers $true -VerificationLevel UseSourceVerification -IsLocal $false -AutodiscoverUrl https://webdir.online.lync.com/Autodiscover/AutodiscoverService.svc/root

clip_image017[10]

Since it worked already, I will go back to the control panel and make sure it is enabled

clip_image018[10]

Next is : Configure your Skype for Business Online tenant for a shared SIP address space

Note:

To configure a shared SIP address space, establish a remote PowerShell session with Skype for Business Online, and then run the following cmdlet:

We’ll have to download skype for business online powershell

https://onedrive.live.com/redir?resid=82488EABA4ACDB15!38849&authkey=!AKW6Ln4Rkn6QuUI&ithint=file%2cexe

After launching the PowerShell module as an administrator I’ll run the following cmdlet

Import-Module SkypeOnlineConnector

clip_image019[8]

Now I’ll connect to my Office 365 tenant

clip_image020[8]

$cred = Get-Credential

$CSSession = New-CsOnlineSession -Credential $cred

Import-PSSession $CSSession -AllowClobber

clip_image021[8]

Now I’ll configure the shared sip address

Set-CsTenantFederationConfiguration -SharedSipAddressSpace $true

From <https://technet.microsoft.com/en-us/library/jj205126.aspx>

clip_image001[22]

To double check my configuration I will see if the SharedSipAddresSpace is enabled or not

Get-CsTenantFederationConfiguration

clip_image002[22]

To double check that the hybrid configuration is setup properly we can use the Skype for business on-premises Hybrid UI wizard from the Home Menu under “Connection to Skype for Business Online”

clip_image003[22]

Using the Skype for Business 2015 User interface to setup Hybrid configuration:

After you sign in it does automatically logs you in and configure the three following options

  1. Federation for the Edge server
  2. Federation with Office 365.
  3. Shared SIP address space.
clip_image004[22]

clip_image005[20]

Now I will configure my DNS Settings as recommended by Microsoft for the Hybrid Integration scenario

DNS Settings

When creating DNS SRV records for hybrid deployments, the records, _sipfederationtls._tcp.<domain> and _sip._tls.<domain>, should point to the on-premises Access Proxy.

From <https://technet.microsoft.com/en-us/library/jj205403.aspx>

  1. Update some DNS records to direct all SIP traffic to Skype for Business on-premises:
  • The lyncdiscover.contoso.com A record to point to the FQDN of the on-premises reverse proxy server.
  • Update the _sip._tls.contoso.com SRV record to resolve to the public IP or VIP address of the Access Edge service of Skype for Business on-premises.
  • Update the _sipfederationtls._tcp.contoso.com SRV record to resolve to the public IP or VIP address of the Access Edge service of Skype for Business on-premises.
  • If your organization uses split DNS (sometimes called “split-brain DNS”), make sure that users resolving names through the internal DNS zone are directed to the Front End Pool.

From <https://support.office.com/en-us/article/Configure-Skype-for-Business-Server-2015-Hybrid-b06ee805-4349-4519-82fb-b06ed57c0bd0>

According to Microsoft’s configuration of the Public DNS, you will have to configure only the SRV records to point to your edge server however, running a simple wireshark on your Skype for business client machine you can notice the following:

clip_image006[20]

Microsoft Lync / Skype client first requires the Lyncdiscover / Lyncdiscoverinternal record in order to see where the user is located… then gets redirected to webdir.online.lync.com which is the Cname value to the Lyncdiscover Cname in the public DNS and tries to login the user through Login.microsoftonline.com then finds no user there and logs in using the SRV eventually in the end as in the below snapshot which I’ve used Wireshark for to monitor the DNS traffic that the Lync Client requests upon login request.

clip_image007[20]

NOTE:

What have me confused here is that Microsoft says only SRV records must be pointing to your On-premises Lync/Skype for Business Edge server.. So you must enter something else other than SIP.domain.com (Which in normal cases might be the common name of your Edge certificate) for the value of the SRV Record since the SIP.domain.com and Lyncdiscover.domain.com must be pointing to Office 365.

I tried using the Public IP address of my Edge server just to check if my on-premises user will connect without any issue however I did have an issue with the Certificate saying “There was a problem verifying the certificate from the server”.

clip_image008[20]

Error:

Luckily the Public certificate that I had on my edge server had multiple SANs (Subject Alternative Names) and one of them was WAC.moh10ly.com which I was intending to use for the WAC Server (Office Web Apps Server) and then I created an A record on my public DNS WAC.moh10ly.com that points to my Edge server’s Public IP address…. although the Wac.moh10ly.com is not a common name but it worked and I was able to federate with Office 365 users and was able to move users from on-premises to office 365 and back to on-premises as demonstrated later in the article.

“When creating DNS SRV records for hybrid deployments, the records, _sipfederationtls._tcp.<domain> and _sip._tls.<domain>, should point to the on-premises Access Proxy.”

From <https://support.office.com/en-us/article/Configure-Skype-for-Business-Server-2015-Hybrid-b06ee805-4349-4519-82fb-b06ed57c0bd0>

clip_image009[14]

Now I have changed all the SRV records to direct to the new A record

clip_image010[14]

And finally deleted the A sip record and created a new CNAME record that points to sipdir.online.lync.com

clip_image011[14]

clip_image012[14]

I have already a user synced from my local AD to the cloud (office 365) that’s not enabled for Skype for business on-premises .. Once this user is synced and have been assigned a license it should be directly enabled for Skype for Business Online and I should be able to sign in to it without any issue.

Note:

In order for both users (homed online and On-premises) to see eachother’s presence the synced user must be enabled on the On-premises Server before moved to the cloud or else the presence and M will fail.

Time to test, I was able to sign in to the Online homed user (admin) and now I’ll be adding the on-premises homed user to the list to check the presence, IM ..etc

clip_image001[24]

Here I added the user admin to my other account Mohammed.hamada and vice versa.

clip_image002[24]

The Presence appears to be working fine for user homed on-premises as it shows when I changed it to “busy, be right back..etc” on the cloud user’s Client however the Office 365 homed user’s presence takes time to change on the on-premises user’s list and the IM doesn’t seem to work properly as messages sometimes doesn’t go through and fail.

Sending a message from the on-premises User (Mohammed Hamada) to (ADMIN)

clip_image003[24]

Now sending an IM from Admin to Mohammed Hamada

clip_image004[24]

To make sure that the issue is not within my on-premises server, I will use a different Skype for Business online account and see if IM work both ways.

This is my other user.. The presence information seems to work properly and now I’ll test the IM

clip_image005[22]

IM between my On-premises and another user on another Office 365 tenant seems to be working fine back and forth as in the below snapshots so the issue might be related to Office 365 tenant which I am using for this test (could be related to trial version)

I am going to open a case with MS and see why this issue happens since my on-premises work fine with other tenants.

clip_image006[22]

clip_image007[22]

Now It’s time to move users from and to cloud and on-premises to check how easy, flexible or hard this process is.

I currently have 2 users, one on cloud and one synced and homed online (Office 365)

clip_image008[22]

In order to move users, you can go to Users tab after the hybrid config is finished and find the user you want to move then click on Actions and chose to move the users to the Skype for Business Online as in the below snapshot

Note:

Before you move the user to Office 365, you must assign license to the user or else the move will fail.

clip_image009[16]

clip_image010[16]

clip_image011[16]

clip_image012[16]

clip_image013[12]

You can move the user back from Office 365 to your on-premises Skype for Business server with the same process exactly except that you’ll have to choose which pool you need to move the user to.

Checking where the user is hosted from Skype for business Management shell

The Hosting Provider will show you where the user is working from now.

clip_image014[12]

clip_image015[12]

clip_image016[12]

Hope this has been helpful

References:

https://technet.microsoft.com/en-us/library/jj204967.aspx

https://technet.microsoft.com/en-us/library/jj205403.aspx

https://technet.microsoft.com/en-us/library/jj205126.aspx

https://technet.microsoft.com/en-us/library/jj204669.aspx

https://support.office.com/en-us/article/Configure-Skype-for-Business-Server-2015-Hybrid-b06ee805-4349-4519-82fb-b06ed57c0bd0

https://channel9.msdn.com/Events/Ignite/2015/BRK4129