I have got a request from a client asking to find out which server(s) is using which domain admin or a highly privileged account as a service.
To find this I already wrote a powershell script that does this, Search the non standard/(Domain only users) and show the services and name of the servers where those accounts are configured on utilizing Remote powershell to do so and the use of a Domain Admin user.
You can refer to this link to see this article by clicking here
Creating the script process:
The same client wanted to also know which of those accounts did authenticate and wanted to know from which server/Computer did the request originate from and to which DC did it go.
I have started thinking of the process of doing so by again utilizing remote PowerShell to check against certain security events on AD to check which user among the Domain admin members did authenticate.
After sometime and with the help of some forums I managed to get script ready which looks in all Domain Controllers for users that are members of the Domain Admin groups who triggered an event ID 4624 and from which Computer did this request came from.
The Script :
# Get domain admin user list
$DomainAdminList = Get-ADGroupMember -Identity 'Domain Admins'
# Get all Domain Controller names
$DomainControllers = Get-ADDomainController -Filter * | Sort-Object HostName
# EventID
$EventID = '4624'
#
# Get only last 24hrs
$Date = (Get-Date).AddDays(-3)
# Limit log event search for testing as this will take a LONG time on most domains
# For normal running, this will have to be set to zero
$MaxEvent = 100
# Loop through Dcs
$DALogEvents = $DomainControllers | ForEach-Object {
$CurDC = $_.HostName
Write-Host "`nSearching $CurDC logs..."
Get-WinEvent -ComputerName $CurDC -FilterHashtable @{Logname='Security';ID=$EventID;StartTime = $Date} -MaxEvents $MaxEvent |`
Where-Object { $_.Properties[5].Value -in $DomainAdminList.SamAccountName } |`
ForEach-Object {
[pscustomobject]@{SourceIP = $_.Properties[18].Value; SamAccountName = $_.Properties[5].Value;Time = $_.TimeCreated;LogonEventLocation = $CurDC}
}
}
$DALogEvents
How to run:
The Script must be run on DC with a privileged account in order to get the write results, The default time interval is set to 3 days but you can choose to increase that.
You can also change the default group where you want to search for members by changing Domain Admin groups to something else.
In office 365 when you’re working on Exchange 2010,2013, 2016 or 2019 in a hybrid environment things might look easy but in a big enterprises where Internet security is something being taken into account very seriously. It might cause many issues that you don’t expect at all.
One of my clients whom I was doing Exchange Migration for had an issue with the Migration. The error was as follows:
Error occurs after Office 365 Exchange online connects to Exchange on-premises 2010 mailbox server
RPR05DG049-db131′, Mailbox server ‘DB3PR05MB0778.eurprd05.prod.outlook.com’
Version 15.1 (Build 466.0).RequestExpiryTimestamp : 03.04.2116 07:42:38
ObjectState : New
Troubleshooting:
To troubleshoot issues, You need to put so many things into account! The architecture of the infrastructure of where you are doing the project is very important and the need of knowing how things are working matters.
Things that could always come in mind and handy are what you will need to start your troubleshooting:
To troubleshoot the MRs, You need to know what kind of error you’re getting and to see this you can use the following powershell after you connect to Office 365 powershell.
The resultant report will reveal the error and shows you where is the exact culprit.
– Disk Latency
– Firewall Configuration (IPS/IDS)
From Exchange 2016 to 2019 or 2013 to 2016 The transient error might be related to MRSProxy or at least this is the case with me 90% of the time. To resolve this issue you will need to change the MRSProxy values on the target server and depending on the error might also be the Source server too.
SOLUTION:
===========
1. Some instability was detected in communications as well as saturation by the size of the link.
2. The procedure to increase the timeout for the service through the file MRSProxy
The Story (Finding Domain Joined Servers Services users)
If you’re wondering which of your servers are using domain joined account or a non regular account like network service or system. You will need to go through every server’s service console and check that one by one but thanks to PowerShell this job was made like a piece of cake.
Requirement
The requirement to run this script is a domain admin account since the PowerShell will require access to other servers using Remote PowerShell using Invoke command and run a Get-WMIObject script to find out those details. So in short I will write the required things for this to work
1- Logged in to Active Directory (In order for AD PowerShell module to run and find computers).
2- Domain admin account (To run the remote PowerShell on other servers and get service details)
3- Firewall for domain joined computers is open (To allow remote PowerShell to work) or have remote PowerShell enabled via GPO.
The Script will also show you the offline (inaccessible servers) and will state those servers as down as you can see in the screenshot below.
The script will also prompt you for a path to save the output. You can enter something like C:\Services.csv as soon as you type the file path and extension it’ll be opened using Notepad.
#Check servers down and get services from the responsive servers
The command will open notepad file with the certificates details in it.
Take a note of all certificate’s thumb print number and open your MMC console
Click File -> add the Certificate then choose Local computer
Navigate to “Trusted Root Certification Authority” store
Check the certificate that their thumb print were shown in the Txt file and remove to the Intermediate Certification authority store.
If you have many certificates in the Trusted root store, you can manage the view and choose “Issued by” and then click on the certificate that the “Issued to” and “Issued by” do not match
double click on it then choose the thumbprint section and try to see if this thumbprint value matches the one in the text and move it to the intermediate store.
When you finish, you must restart the servers one by one in order to resolve this issue and then you will notice that the error is gone and that services are back to normal state
Open ADSIEDIT and look in the following snapshot. Open Configuration for your DC
Collapse the menu and click on Services
Click on RTC Service
Click on Global Settings and on the right pane look if there’s any duplicated entries and remove them.
As you can see on my right pane I have 2 duplicated (msRTCSIP-EdgeProxy) and I’m going to remove one of them and see if I can publish my topology or not. But before that I will have to make sure that I export the entry that I wanna delete.
I right clicked on the last value and deleted it and here how it became now.
Now I will try to publish my topology and see what happens, my topology publishing failed with a new error this time.
I will have to go and check where’s this coming from, since it mentions TrustedService. I will go look in the trusted service
This is not going to be easy, as you need to becareful where you look .. You will need to make sure that you’re looking at the right FQDN
Here I could find the value MRAS for the FQDN Edge server
So I looked here and found 2 identical entries with a different (CN) if you scroll down you will see that the GruuId is the same, FQDN is the same, port is the same.
Let’s delete one of them and see again if we can publish our topology, So I deleted the one that starts with {b344}
I will do this using the Lync Powershell, you can see below that the Topology was published successfully.
To resolve the warning you will have to issue the cmdlet Enable-CsAdForest after the Enable-CsTopology
Skype for Business Edge server deployment and Hybrid integration with Skype for Business Online
In the last Skype for Business post I have upgraded my Lync 2013 to Skype for Business (Click here to go to that post). in this article I am going to install Edge server for Skype for Business to the same Lync Environment where I have done the Upgrade to Skype for Business.
Configuring Edge Server
Setup NETBIOS
In order to configure Skype 4 Business Edge, we’ll have to change the Netbios to give it the name of our Domain but we won’t join it to the domain.
Setup NICs
Edge Server must have 2 NICs, one Local NIC will point out to the Front end server but must not have Default gateway so traffic can only flow through the DMZ out to the internet and back in. but still it must be able to ping to the FE from Edge and vice versa.
The DMZ network could have a single DMZ address (Public Address to be pointing to) or three DMZ addresses for public IP addresses with standard https ports.
Configure Hostnames
Edit the Edge server’s host file to include Lync FE and DC’s IP addresses and Hostname
Install Prerequisites
Microsoft .Net Framework 3.5
Now I will go back to Skype for Business FE server, I’ll launch the topology builder and add new Edge server
I will add the first Edge pool which contains of a single Edge server
Next, you will have to choose if you want to enable federation with partners or other service providers …e.g. (Google)
I am intending to use a single Public IP address with a different ports (nonstandard) since this is a lab. For production use it’s recommended to have 3 public IP addresses, One is for Access Edge, AV and WebConf services.
Next I will choose the last option which says that the Edge pool is translated by NAT. I will configure my firewall to NAT ports to the Edge’s DMZ IP addresses from the Public so I am choosing this option.
This is the FQDN’s the default configuration .. It’ll only use a single FQDN for all services if you’re going to use a single public IP address with a different ports.
IMPORTANT NOTE
When you use a single IP address with a different ports, the Access Edge port will normally change to 5061 (Not 443 like in the _sip._tls.domain.com) SRV record which will cause failure if you forgot to change this port to match the one in your Topology’s Access Edge settings.
Next I’ll have to enter my Edge server’s Local IP address.
Next I will be asked to enter the DMZ’s IP address which the wizard calls (Private External IP address)
Here I am going to place the NAT IP address which is my Public IP address.
Next I’ll have to choose which Lync FE pool will be used as the next hop to the Edge pool. In this case I’ll be choosing my main pool since the second is only for resilience purpose.
Then I’ll associate the mediation pool for Edge server for external media traffic. I can assign both in this case.
Now I’ll click on Finish and right click on the Site name’s properties to enable the SIP federation and XMPP federation then Publish the topology.
Now I will setup Azure Active Directory Sync on my DC server in order to sync the required users for the test purpose.
My domain is adeo.local so I want to change the UPN for users to match the synced domain. (Adeo-office365.ga) and moh10ly.com
Installing Azure Active Directory Sync
Now I will install the prerequisites which consist of the following
Net framework 4.5.2 is required for AADS but it’s already installed on my server
Next I will install Microsoft Online Service Sign in assistant
Next I will install Azure AD Module
Finally Azure AD Sync
Before moving forward, I’ll have to go to the Office 365 portal and activate DirSync
Then use a global admin credentials from O365.
Adding the forest using an enterprise admin user account
Due to the fact that my domain adeo-office365.ga’s public dns host doesn’t have SRV configuration because it’s hosted by the famous free domain service (Freenom) so I’ll have to add my original domain moh10ly.com as Lync (S4B) requires SRV records to point to the on-premises lync.
I will only sync one OU, so I will untick the Sync now box and click on Finish
I will go to the following path
“C:\Program Files\Microsoft Azure AD Sync\UIShell” and create a shortcut for the GUI application of AADS on the desktop
“C:\Program Files\Microsoft Azure AD Sync\UIShell\miisclient.exe”
To get this GUI app to work, you will have to sign out of your account and sign back in as your username will be added to the local administrators and have the authority to open it
Log off, log back in
Next I will go to the connectors tab and double click on the ADDS connector (Adeo.local)
I will go to the Configure Directory Partitions and under Credentials I’ll choose “Alternate credentials for this directory partition” then enter my on-premises AD Enterprise admin credentials
I’ll click on Containers
I’ll untick the DC=Adeo,Dc=Local box and only choose Dirsync OU then click OK and apply
Before I start syncing my AD , I will go to Skype for Business Server and add my domain moh10ly.com as a SIP domain
Next I am going to change the FQDN of the SIP access edge for public domain to moh10ly.com and the default port for the Access Edge to 443 and publish the topology
I needed to finally check if all my FE servers are replicating. So then I can move to Edge server to install Lync components
On the Edge server, I’ll use ISO for Skype 4 business to install the setup
First thing I’ll install the local Configuration Store
I’ll click on Run and then I’ll be asked to import the configuration file which I’ll must export from Lync FE (Skype 4 b FE) server
In this case, I’ll go to Lync FE and open Lync Management shell and enter the following Cmdlet
Export-CsConfiguration -FileName c:\top.zip
This cmdlet will export a file to the root C drive . I’ll copy this file to the edge server.
I’ll click next to continue, this should start installing the local store
Next I’ll request a certificate for Internal NIC For edge server
Configure Certificate
I’ll take the CSR (Certificate sign request) code and get a certificate from my local CA
I’ll open MMC and add Certificates console and import the PKCS certificate
After importing the certificate I’ll assign it to the internal NIC by clicking on Assign to the Edge Internal
Once we assign the certfiicate to the internal edge. The replication service for Edge and FE will start working
Now I’ll import my Public Certificate to Edge Server’s DMZ NIC
I already imported my public certificate, now I’ll go to the S4B wizard and assign it there
Unlike IN lync 2013 when you Click on Start service in the Wizard all services start on their own but on Skype for business you ‘ll have to start the services manually by yourself.
So Instead I used the service console to start the services.
Now I’ll go back to the FE And enable remote connectivity to Skype for Business from outside and make sure that replication works fine by checking the Topology or from cmdlet
After the replication is finished, I was able to login remotely with my Skype for Business on-premises accounts.
—
Setting up Hybrid integration with Skype online for Business (O365)
In order to allow Hybrid environment to function properly, we’ll have to federate our Skype for Business on-premises’s Edge server as Microsoft says below
Federation allows users in your on-premises deployment to communicate with Office 365 users in your organization. To configure federation, run the following cmdlets in the Skype for Business Server Management Shell:
Next cmdlet will create a new public federated provider for skype for business online.. However it already exists by default as in the below snapshot but just to avoid any issues I will delete the default provider from control panel and recreate it again.
I’ll delete the hosted provider “Skype for Business Online”
I’ll try the cmdlet again after deleting the provider ..
To double check my configuration I will see if the SharedSipAddresSpace is enabled or not
Get-CsTenantFederationConfiguration
To double check that the hybrid configuration is setup properly we can use the Skype for business on-premises Hybrid UI wizard from the Home Menu under “Connection to Skype for Business Online”
Using the Skype for Business 2015 User interface to setup Hybrid configuration:
After you sign in it does automatically logs you in and configure the three following options
Federation for the Edge server
Federation with Office 365.
Shared SIP address space.
Now I will configure my DNS Settings as recommended by Microsoft for the Hybrid Integration scenario
DNS Settings
When creating DNS SRV records for hybrid deployments, the records, _sipfederationtls._tcp.<domain> and _sip._tls.<domain>, should point to the on-premises Access Proxy.
Update some DNS records to direct all SIP traffic to Skype for Business on-premises:
The lyncdiscover.contoso.com A record to point to the FQDN of the on-premises reverse proxy server.
Update the _sip._tls.contoso.com SRV record to resolve to the public IP or VIP address of the Access Edge service of Skype for Business on-premises.
Update the _sipfederationtls._tcp.contoso.com SRV record to resolve to the public IP or VIP address of the Access Edge service of Skype for Business on-premises.
If your organization uses split DNS (sometimes called “split-brain DNS”), make sure that users resolving names through the internal DNS zone are directed to the Front End Pool.
According to Microsoft’s configuration of the Public DNS, you will have to configure only the SRV records to point to your edge server however, running a simple wireshark on your Skype for business client machine you can notice the following:
Microsoft Lync / Skype client first requires the Lyncdiscover / Lyncdiscoverinternal record in order to see where the user is located… then gets redirected to webdir.online.lync.com which is the Cname value to the Lyncdiscover Cname in the public DNS and tries to login the user through Login.microsoftonline.com then finds no user there and logs in using the SRV eventually in the end as in the below snapshot which I’ve used Wireshark for to monitor the DNS traffic that the Lync Client requests upon login request.
NOTE:
What have me confused here is that Microsoft says only SRV records must be pointing to your On-premises Lync/Skype for Business Edge server.. So you must enter something else other than SIP.domain.com (Which in normal cases might be the common name of your Edge certificate) for the value of the SRV Record since the SIP.domain.com and Lyncdiscover.domain.com must be pointing to Office 365.
I tried using the Public IP address of my Edge server just to check if my on-premises user will connect without any issue however I did have an issue with the Certificate saying “There was a problem verifying the certificate from the server”.
Error:
Luckily the Public certificate that I had on my edge server had multiple SANs (Subject Alternative Names) and one of them was WAC.moh10ly.com which I was intending to use for the WAC Server (Office Web Apps Server) and then I created an A record on my public DNS WAC.moh10ly.com that points to my Edge server’s Public IP address…. although the Wac.moh10ly.com is not a common name but it worked and I was able to federate with Office 365 users and was able to move users from on-premises to office 365 and back to on-premises as demonstrated later in the article.
“When creating DNS SRV records for hybrid deployments, the records, _sipfederationtls._tcp.<domain> and _sip._tls.<domain>, should point to the on-premises Access Proxy.”
Now I have changed all the SRV records to direct to the new A record
And finally deleted the A sip record and created a new CNAME record that points to sipdir.online.lync.com
I have already a user synced from my local AD to the cloud (office 365) that’s not enabled for Skype for business on-premises .. Once this user is synced and have been assigned a license it should be directly enabled for Skype for Business Online and I should be able to sign in to it without any issue.
Note:
In order for both users (homed online and On-premises) to see eachother’s presence the synced user must be enabled on the On-premises Server before moved to the cloud or else the presence and M will fail.
Time to test, I was able to sign in to the Online homed user (admin) and now I’ll be adding the on-premises homed user to the list to check the presence, IM ..etc
Here I added the user admin to my other account Mohammed.hamada and vice versa.
The Presence appears to be working fine for user homed on-premises as it shows when I changed it to “busy, be right back..etc” on the cloud user’s Client however the Office 365 homed user’s presence takes time to change on the on-premises user’s list and the IM doesn’t seem to work properly as messages sometimes doesn’t go through and fail.
Sending a message from the on-premises User (Mohammed Hamada) to (ADMIN)
Now sending an IM from Admin to Mohammed Hamada
To make sure that the issue is not within my on-premises server, I will use a different Skype for Business online account and see if IM work both ways.
This is my other user.. The presence information seems to work properly and now I’ll test the IM
IM between my On-premises and another user on another Office 365 tenant seems to be working fine back and forth as in the below snapshots so the issue might be related to Office 365 tenant which I am using for this test (could be related to trial version)
I am going to open a case with MS and see why this issue happens since my on-premises work fine with other tenants.
Now It’s time to move users from and to cloud and on-premises to check how easy, flexible or hard this process is.
I currently have 2 users, one on cloud and one synced and homed online (Office 365)
In order to move users, you can go to Users tab after the hybrid config is finished and find the user you want to move then click on Actions and chose to move the users to the Skype for Business Online as in the below snapshot
Note:
Before you move the user to Office 365, you must assign license to the user or else the move will fail.
You can move the user back from Office 365 to your on-premises Skype for Business server with the same process exactly except that you’ll have to choose which pool you need to move the user to.
Checking where the user is hosted from Skype for business Management shell
The Hosting Provider will show you where the user is working from now.
This article guides you through the steps of doing an in-place upgrade from Lync 2013 to Skype for business. I am copying the article as is from my lab with all the errors that I have been through to give you a real experience feed back of what is this like.
You might get issues that you have never expected, but resolving them is not that hard and if you have any issues please don’t hesitate to leave a comment and I will get back to help you.
Prerequisites
Extensible Chat Communication Over SIP protocol (XCCOS)
Prerequisite not satisfied: Internet Information Services (IIS) must be installed before attempting to install this product.
Prerequisite not satisfied: The following Internet Information Services (IIS) role services must be installed before attempting to install this product: Static Content, Default Document, HTTP Errors, ASP.NET, .NET Extensibility, Internet Server API (ISAPI) Extensions, ISAPI Filters, HTTP Logging, Logging Tools, Tracing, Client Certificate Mapping Authentication, Windows Authentication, Request Filtering, Static Content Compression, Dynamic Content Compression, IIS Management Console, IIS Management Scripts and Tools
Prerequisite not satisfied: Before you install Skype for Business Server 2015, you must install an update for Windows Server 2012 R2. For details about the update, see Microsoft Knowledge Base article 2982006, “IIS crashes occasionally when a request is sent to a default document in Windows 8.1 or Windows Server 2012 R2” at http://go.microsoft.com/fwlink/?LinkId=519376
Prerequisite not satisfied: Before you install Skype for Business Server 2015, you must install Microsoft ASP.NET 4.5 by using the Add Roles and Features Wizard in Windows Server 2012 Server Manager. Install the ASP.NET 4.5 role service of the Web Server (IIS) role.
Prerequisite not satisfied: Before you install Skype for Business Server 2015, you must install Microsoft Windows Communication Foundation Activation by using the Add Roles and Features Wizard in Windows Server 2012 Server Manager. Install WCF Services and HTTP Activation, which are included with the Microsoft .NET Framework 4.5 feature.
After the restart we will apply the update of the databases which in my case is going to be the FQDN of the FE server since it’s standard version and not Backend server.
My SQL Server version is SP1 so I don’t need to upgrade it to SP2
Step 6- In-place Upgrade for Skype For Business
In order to do the in-place upgrade, we’ll need to use a machine that doesn’t have Lync 2013 to install the new Topology builder and do the upgrade process
On a different Machine that’s joined to the same domain, I will run the prerequisites script and restart the machine. then I’ll load the Skype for business ISO and install
D:\Setup\amd64\Setup.exe
We’ll now press on Installing Administrative tools
Now in order to continue we’ll have to open the topology builder in order to upgrade our Lync 2013 topology
I’ll open the topology builder and save the topology file somewhere
Once the topology is open, I’ll navigate to the Standard FE Servers and right click on my main server to upgrade
I’ll click on Upgrade to Skype for Business Server 2015…
As soon as you press Yes, the Frontend server that you selected will be moved under the Skype For Business Server 2015 tab as you can see below.
Since I have two FE servers (FE and SBS) I will be upgrading them both but not in the same time not not fall into any errors, so I will publish the topology and see what happens.
We’ll check what do we need to do now in order to upgrade the servers, here is what we’ll do.
Import existing normalization rules from the previous Skype for Business Server deployment. If you want to keep your existing normalization rules you will need to import them using the Import-CsCompanyPhoneNormalizationRules cmdlet. If you have separate normalization rules for each pool then you will need to run the command for each set.
To perform an in-place upgrade of your Skype for Business Server, you’ll need to do the following, in order:
(1) Stop the Skype for Business services on all of the servers that you are upgrading;
(2) Run Skype for Business Server setup (Setup.exe) on all of the servers you are upgrading;
(3) Start the Skype for Business services on all of the servers you upgraded. To start the services in a Front End pool, connect to one of the servers in the pool and run the Start-CsPool cmdlet. All the servers in the pool should be running Skype for Business Server before you use the Start-CsPool cmdlet. To start the services in all other pools (e.g. Edge pool, Mediation pool), run the Start-CsWindowsService cmdlet on every server in the pool;
Server FQDN: lyncfe01.adeo.local, Pool FQDN: lyncfe01.adeo.local
On Lync FE 01 I’ll stop all the services using Stop-cswindowsservice
Now on the same server I’ll load the Skype4B ISO and start the setup
D:\Setup\amd64\Setup.exe
Started at 1:40pm
NOTE:
The required time for the upgrade process is estimated around 75-90 Minutes for each FE Server.
So Exchange 2016 preview version came on MSDN and I decided to give it a try along with the DAG …
Previously in Exchange 2013 I used to have an issue with the fast search on DAG as in some cases it used to stop and cause the original database and copy to report not healthy.
Here I wanted to Install exchange 2016 on new windows edition along with configuring DAG and observe the database’s indexing status.
So to start, I used the available Microsoft Technet related to Exchange 2016.
I’ll launch Powershell as adminsitrator an start by installing the requested software
Before moving on you will have to fulfill the Software Prerequisites which is attached below.. It’s the same as in Exchange 2013.
Now I will start the Exchange Installation from the Setup, You can follow the setup till the end as in the following screenshots:
Now I will create DAG and replicate DBs and notice FastSearch logs
Since this is a LAB and I only have 2 nodes (DAG must have an odd number for Failover), so I am going to use the DC server as my FSW (Which is highly not recommended for Production Environment).
In order for the FSW on DC to work, you will have to add your DC to the Exchange Trusted Subsystem group
Here I added the DC as a member of the group
Normally file server feature is already added to the server by default, but to make sure I’ll run the following command
It’s already there
Now I’ll go back to Exchange servers and add a second NIC for Replication..
I usually rename each NIC so I know which is which, the default NIC belongs to the MAPI traffic and the other one is the replication NIC.
I will configure the Replication NICs on both exchange servers to disable the “Register this connection’s addresses in DNS”
Checking ping between Exchange servers on the Replication NIC
Now I will create the CNO Object (Cluster Name Object) in the Active directory for the DAG
I will disable the object
Double click on the DAG object and go to Security tab and add Exchange servers
Now configure the security for the Exchange members to full
Apply and close…
Now on the DC I will create the FSW’s folder and give it full permission to the Trusted Subsystem group and exchange servers
Click apply and go back to EAC and I’ll start configuring the DAG
Microsoft says that one of the enhancements that have been added to Exchange 2016 is that DatabaseAvailabilityGroupIpAddresses is no longer required when creating a DAG.
By default, the failover cluster will be created without an administrative access point, as this is the recommended best practice.
So in this case we won’t need to assign any IP address to the DAG…
I’ll click on save and see what happens
Navigating to the administrators group on AD, The Exchange subsystem group is not added so I’ll add it.
Now I will add Exchange servers as members to the DAG
Upon adding the Exchange members to DAG I got the following error
A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering isn’t installed on ‘EXCH2K16.test.com’.. [Server: EXCH2K16.test.com] error A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering isn’t installed on ‘EXCH2k1602.test.com’.. [Server: EXCH2k1602.test.com]
So I checked the following, 1- Firewall 2- CNO’s security settings.
Error occurs due to firewall being enabled on the DC (Where the FSW is )
I disabled the firewall and gave full permission to the Exchange trusted system to the DAG object
After that I signed out of Exchange servers, signed back in.. Deleted DAG and recreated it… that didn’t work either
Tried using Exchange management shell but it didn’t work too
Checking the log coming in the description, I find out the log is complaining about not finding DAG while trying to resolve it.
Also the log says that it has installed Failover cluste rbut still the cluster can’t find FQDN called DAG.
So I will have to configure DAG in the dns and give it an IP of my first Exchange server
Checking DAG resolving from Exchange server
As the log says, restart is required after installing failover cluster so I’ll restart Exchange servers and then retry to add Exchange servers to the dag.
After restarting the server, It seems that things are working
The second server gave the following error
The Microsoft Exchange Replication service does not appear to be running on “EXCH2k1602”. Make sure that the server is operating, and that the services can be queried remotely.
Apparently the error is correct, After restarting the server it turned out that most of the second Exchange server’s services were not working..
To be honest I didn’t ask myself why did not the services started since I am using a preview version of Exchange 2016 and Windows as well so I manually started all the services.
Interestingly while checking services, I noticed new services e.g. (DAG Management, Compliance Audit, Notifications broker)
After starting the services, now I tried to add the second server again to the DAG.
So eventually, DAG doesn’t need an IP address but still a DNS value needs to be created for the NCO object and needs to have an IP assigned to it which will be the Exchange server IP address..
Next: I will add a database copy and see how it’s improved and do I need to restart the IS service as in Exchange 2013.
I will leave all the default values and add the second server for the database to be copied on. Unlike Exchange 2013 in most of the times the database would fail first and gives an error ..
In 2016 it starts directly seeding the database to the second Exchange server that’s member of the DAG.
On the second Server where the database has been copied to, I checked the Logs and Fastsearch was throwing errors as usual since database logs are not copied … as soon as the database logs finished copying the fastsearch will return ok and the database will appear as Healthy in EAC.
Fastsearch finally reported that indexing started on the newly copied DB.
The database copy should now report healthy in the EAC.
Hope you find this useful. In the next article I will publish the Exchange server online to check the rest of the functionalities.
To setup UM between Exchange and Skype for business server, the most important step is how you configure the Certificates between both servers in order for them to trust each other.
For that you don’t have to use a public Certificate but rather an internal CA certificate that has its root certificate installed on all of the server where you intend to deploy the UM. (Exchange, S4B Servers..etc.).
To claim this certificate, the easiest step would be to get the CSR from Skype for Business’s Deployment Wizard
Run Deployment Wizard and click on the “Install or Update skype for business Server system”
Then click on step 3 (request, install or assign Cert)
I already have certificate deployed for S4B service but I’ll request CSR again to get one trusted certificate for both Exchange and S4B.
I will tick only the services that matters as in the below screenshot (Server default and Web services internal) later also will be used for OWA integration with UM.
Click on Request
Click on Advanced
Next
I’ll continue next until I’ve got to the important part which is “Name and Security settings” I’ll need to tick the “Mark the certificate’s private key as exportable” since we’ll export the certificate to Exchange servers
Next, adding a gateway to the UM (NOTE: If configured incorrect, will cause the service not to start and errors with event ID (1057, 4999,1430, 1038) will appear.
Time to configure Gateway
In the gateway I’ll add my PBX (AsteriskNow) and place my already configured UMDP
When you create the dial plan, Exchange automatically creates a new UM mail policy along with it and it also generates a name that’s related to the Dial plan
In order to see this policy, you will have to double click on the new dial plan to view it and you can also change the policy in it .. Which I’m going to apply for the length of the policy to make it shorter
Double click on the Mailbox policy and navigate to Pin Polices and change it to the length you want to allow
Configure Auto Attendant
Set the AA as how you want it to be configured and make sure you add the full E.164 format as it won’t accept otherwise.
Click Save to continue
Now time to configure OVA (Outlook voice access)
Subscriber Access
If you want to configure Outlook Voice Access (OVA) , sometimes also referred to as Subscriber Access, click on the Configure button. Select Outlook Voice Access in the left hand menu and enter the telephone number you want to use to access OVA. This must be in the E.164 notation.
To do so click on Configure
To assign the new dial plan to the UM services, both on the Client Access Server (UM Call Router) as well as on the Mailbox server. In an Exchange Management Shell windows enter the following commands:
To configure the UM Service to be used with Skype for Business Server. Microsoft has a script that will create and configure all necessary components. This scripts is located in the scripts directory C:\Program Files\Microsoft\Exchange Server\V15\Scripts.
Run the following CMDLET
CD $ExScripts
.\ExchUCUtil.ps1
The first time you setup this script it’ll detect the Dial plan and set it up with Skype for Business Server
It will show that no setting has changed but the fact that the dial plan is showing here Not found means that there something has changed .. You’ll notice that if you run the same script again.
Let’s try it again
Here you can see that the dial plan has been assigned to the S4B Front end server.
This script performs the following:
Grants Skype for Business Server permission to read Exchange UM Active Directory components, specifically, the SIP URI dial plan that was created in the first step;
Creates a UM IP gateway for each Skype for business Server pool that hosts users who will be enabled for Enterprise Voice;
Create an Exchange UM hunt group for each UM IP gateway. The hunt group pilot identifier will be the name of the dial plan associated with the corresponding UM IP gateway. The hunt group must specify the UM SIP dial plan used with the UM IP gateway.
When the script has run you’ll see a new UM IP Gateway appear in the EAC. Since this script not only creates the UM IP Gateway but also sets the necessary permissions the UM IP Gateway was not created manually in the first step.
Next we’ll go to Skype for Business FE server and then run the OcsUmUtil.exe tool which creates the contact objects for Outlook Voice Access and for the auto attendants. This tool can be found in C:\Program Files\Common Files\Skype for Business Server 2015\Support
I’ll right click the file to run it as administrator
Click on Load Data
Select the SIP dial plan and click ADD
Click OK
Right after configuring this your Voice mail should be enabled once you enable your user for it
After I enable user for UM and assign a valid dialplan .. Now I can see the user has got his Voice Mail option available.
If you’re looking for an quick way to let all your users easily add all Skype for Business users to their list after migration from Lync 2010/2013/Skype4business to Office 365 Skype for Business then please follow these steps ..
In order to do so, you will have to have DirSync (Azure AD Sync) installed and functioning properly.
First step: Add a group to AD
On Local AD create a Universal Distribution group as following
Note:
The group must have an e-mail address entered in the Email field otherwise it won’t show up in Lync Client list when you search.
Go to Members tab and add all the users that you are planning to Enable on Skype4Business.
Apply and close the group.
Go to DirSync
Force the Sync
Make sure that group has been Synced.
In office 365. You can check If the group is there or not by simply navigating to the Groups tab on the left pane.
Now Open Lync 2013 or Skype 4 Business client and search for this group by email
Right click the group and click Add to contacts
As soon as you add the group, all the members will come beneath it right away.
![clip_image001[4]](https://www.moh10ly.com/wp-content/uploads/2019/12/clip_image0014.png "clip_image001[4]")
![clip_image001[6]](https://www.moh10ly.com/wp-content/uploads/2019/12/clip_image0016.png "clip_image001[6]")
![clip_image001[8]](http://lh3.googleusercontent.com/-JVkzP-XEz9w/VdYhkuAGs5I/AAAAAAAAQ5Y/DjnZoSDpPx8/clip_image001%25255B8%25255D_thumb.png?imgmax=800)
![clip_image002[8]](http://lh3.googleusercontent.com/-FVMSyPjzqGs/VdYhoTQELrI/AAAAAAAAQ5o/CMEL4EogEss/clip_image002%25255B8%25255D_thumb.png?imgmax=800)
![clip_image003[8]](http://lh3.googleusercontent.com/-IXRzI-X9G18/VdYhsrCEunI/AAAAAAAAQ54/W9dkBO8IgX0/clip_image003%25255B8%25255D_thumb.png?imgmax=800)
![clip_image004[8]](http://lh3.googleusercontent.com/-cEe9EaUzjmo/VdYhxQG3qxI/AAAAAAAAQ6I/0gtGN0MSA3E/clip_image004%25255B8%25255D_thumb.png?imgmax=800)
![clip_image001[10]](http://lh3.googleusercontent.com/-n2NU6FBkm70/VdYh6ekLIxI/AAAAAAAAQ6o/nfVnJwXl2fY/clip_image001%25255B10%25255D_thumb.png?imgmax=800)
![clip_image002[10]](http://lh3.googleusercontent.com/-eLE2NzM2DkM/VdYh_q-EiLI/AAAAAAAAQ64/Hj03Kuovdhs/clip_image002%25255B10%25255D_thumb.png?imgmax=800)
![clip_image003[10]](http://lh3.googleusercontent.com/-pWVj4HU7CT8/VdYiEozxHSI/AAAAAAAAQ7I/yxPFOVitglg/clip_image003%25255B10%25255D_thumb.png?imgmax=800)
![clip_image004[10]](http://lh3.googleusercontent.com/-CofvDYhqSE8/VdYiKabZRzI/AAAAAAAAQ7Y/2zW5wCS9rxA/clip_image004%25255B10%25255D_thumb.png?imgmax=800)
![clip_image005[8]](http://lh3.googleusercontent.com/-eps2XJ1RnJg/VdYiORUPdHI/AAAAAAAAQ7o/jKXTOS0ntls/clip_image005%25255B8%25255D_thumb.png?imgmax=800)
![clip_image006[8]](http://lh3.googleusercontent.com/-Ll6ASulX9wU/VdYiTf62hLI/AAAAAAAAQ74/uKqFwHf-mgI/clip_image006%25255B8%25255D_thumb.png?imgmax=800)
![clip_image007[8]](http://lh3.googleusercontent.com/-LL0I9YnKk4o/VdYiZImKXlI/AAAAAAAAQ8I/a7cizhc7ZCo/clip_image007%25255B8%25255D_thumb.png?imgmax=800)
![clip_image001[12]](http://lh3.googleusercontent.com/-K7Df_zq_eqM/VdYioumjC0I/AAAAAAAAQ8o/UUg2lCwdhLw/clip_image001%25255B12%25255D_thumb.png?imgmax=800)
![clip_image002[12]](http://lh3.googleusercontent.com/-FOmoic6xGO8/VdYitl5tzFI/AAAAAAAAQ84/RQLn2vVE52c/clip_image002%25255B12%25255D_thumb.png?imgmax=800)
![clip_image003[12]](http://lh3.googleusercontent.com/-SVhMM9754yw/VdYi8q2tFXI/AAAAAAAAQ9I/wo5YMGzSW8I/clip_image003%25255B12%25255D_thumb.png?imgmax=800)
![clip_image004[12]](http://lh3.googleusercontent.com/-fbLvjtfoFN4/VdYjGJ1vCqI/AAAAAAAAQ9Y/s6zJzH8bLTw/clip_image004%25255B12%25255D_thumb.png?imgmax=800)
![clip_image005[10]](http://lh3.googleusercontent.com/-C3z9Y6ipqe0/VdYjRmDAcbI/AAAAAAAAQ9o/eiUiUM5b9xc/clip_image005%25255B10%25255D_thumb.png?imgmax=800)
![clip_image006[10]](http://lh3.googleusercontent.com/-9l_igww_DuI/VdYjbi9xoaI/AAAAAAAAQ94/mWH6MtrjcYo/clip_image006%25255B10%25255D_thumb.png?imgmax=800)
![clip_image007[10]](http://lh3.googleusercontent.com/-8F3qp5qucSw/VdYjjTgXXjI/AAAAAAAAQ-I/IwxEYcbtNt4/clip_image007%25255B10%25255D_thumb.png?imgmax=800)
![clip_image008[10]](http://lh3.googleusercontent.com/-Y3nckkNQ-fo/VdYjrQH1uKI/AAAAAAAAQ-Y/ZrJt0AjyYv0/clip_image008%25255B10%25255D_thumb.png?imgmax=800)
![clip_image009[8]](http://lh3.googleusercontent.com/-dXk-iOx6lio/VdYj2zytQyI/AAAAAAAAQ-o/shKzm7feJF4/clip_image009%25255B8%25255D_thumb.png?imgmax=800)
![clip_image010[8]](http://lh3.googleusercontent.com/-FkiEJpRN3lk/VdYkC0F4s6I/AAAAAAAAQ-4/HFqoLAKeNlA/clip_image010%25255B8%25255D_thumb.png?imgmax=800)
![clip_image011[8]](http://lh3.googleusercontent.com/-F0OJH2Xft1o/VdYkMBMFHaI/AAAAAAAAQ_I/cw3UksQU2B0/clip_image011%25255B8%25255D_thumb.png?imgmax=800)
![clip_image012[8]](http://lh3.googleusercontent.com/-dyBMGLqyTLI/VdYtdNIIqHI/AAAAAAAAQ_g/Pl0LuF60fls/clip_image012%25255B8%25255D_thumb.png?imgmax=800)
![clip_image001[14]](http://lh3.googleusercontent.com/--9eT1-XKjjk/VdYthjxiiQI/AAAAAAAAQ_w/dzx42bHFxKE/clip_image001%25255B14%25255D_thumb.png?imgmax=800)
![clip_image002[14]](http://lh3.googleusercontent.com/-HCpFLGYNVMc/VdYtlU4quDI/AAAAAAAARAA/btkogyaiDqM/clip_image002%25255B14%25255D_thumb.png?imgmax=800)
![clip_image003[14]](http://lh3.googleusercontent.com/-M58J612IJYg/VdYtqCTWdpI/AAAAAAAARAQ/r_4p-4q4sps/clip_image003%25255B14%25255D_thumb.png?imgmax=800)
![clip_image004[14]](http://lh3.googleusercontent.com/-wSI-7mvPMr4/VdYttqXtisI/AAAAAAAARAg/DnGzOrDGnA8/clip_image004%25255B14%25255D_thumb.png?imgmax=800)
![clip_image005[12]](http://lh3.googleusercontent.com/-0Ge3CltnB_k/VdYtxh0WWBI/AAAAAAAARAw/J3v24wdlCqk/clip_image005%25255B12%25255D_thumb.png?imgmax=800)
![clip_image006[12]](http://lh3.googleusercontent.com/-suuE6vyARvQ/VdYt1u30PFI/AAAAAAAARBA/myxd7CWA1vY/clip_image006%25255B12%25255D_thumb.png?imgmax=800)
![clip_image007[12]](http://lh3.googleusercontent.com/--o4XAN0saAU/VdYt7qmcVZI/AAAAAAAARBQ/3ulUN4mgWG0/clip_image007%25255B12%25255D_thumb.png?imgmax=800)
![clip_image008[12]](http://lh3.googleusercontent.com/-FzGFXha1SzE/VdYt_0g2CDI/AAAAAAAARBg/5NqbbCJh2UY/clip_image008%25255B12%25255D_thumb.png?imgmax=800)
![clip_image001[16]](http://lh3.googleusercontent.com/-YAq1_Li5HLA/VdYuENY4awI/AAAAAAAARBw/ON3X_TtevgM/clip_image001%25255B16%25255D_thumb.png?imgmax=800)
![clip_image002[16]](http://lh3.googleusercontent.com/-OFiNW5iJZ_0/VdYuHu78oAI/AAAAAAAARCA/pSgAJu7PQrE/clip_image002%25255B16%25255D_thumb.png?imgmax=800)
![clip_image003[16]](http://lh3.googleusercontent.com/-o23CbotvIa8/VdYuLBeJItI/AAAAAAAARCQ/H1AQx8NhLLg/clip_image003%25255B16%25255D_thumb.png?imgmax=800)
![clip_image004[16]](http://lh3.googleusercontent.com/-pEdTSxqyp24/VdYuPTUkftI/AAAAAAAARCg/MXF8B5qoogc/clip_image004%25255B16%25255D_thumb.png?imgmax=800)
![clip_image005[14]](http://lh3.googleusercontent.com/-xr7iM7Y99n4/VdYuS3oH9fI/AAAAAAAARCw/Tpqsz9cpxKA/clip_image005%25255B14%25255D_thumb.png?imgmax=800)
![clip_image006[14]](http://lh3.googleusercontent.com/-_clCZutXRJQ/VdYuYZh_5xI/AAAAAAAARDA/pHouOKcBiBU/clip_image006%25255B14%25255D_thumb.png?imgmax=800)
![clip_image007[14]](http://lh3.googleusercontent.com/-l2D-du4nIEk/VdYuckIlTcI/AAAAAAAARDQ/6Ij6qtW6sX4/clip_image007%25255B14%25255D_thumb.png?imgmax=800)
![clip_image008[14]](http://lh3.googleusercontent.com/-7lh25ZwGM74/VdYugmuzazI/AAAAAAAARDg/uYa1TVF4deE/clip_image008%25255B14%25255D_thumb.png?imgmax=800)
![clip_image001[18]](http://lh3.googleusercontent.com/-_Eo1ySRhc7w/VdYulI3X6iI/AAAAAAAARDw/nm7auzI_NdI/clip_image001%25255B18%25255D_thumb.png?imgmax=800)
![clip_image002[18]](http://lh3.googleusercontent.com/-7f9VGDbVlqc/VdYupO894vI/AAAAAAAAREA/hKnRU-QNoEI/clip_image002%25255B18%25255D_thumb.png?imgmax=800)
![clip_image003[18]](http://lh3.googleusercontent.com/-6d3IYHIyLAc/VdYuvBFO7YI/AAAAAAAAREQ/Ley-JsHBKmw/clip_image003%25255B18%25255D_thumb.png?imgmax=800)
![clip_image004[18]](http://lh3.googleusercontent.com/-jE1diCbah_I/VdYuyXXjRgI/AAAAAAAAREg/fTGdJIo7XmE/clip_image004%25255B18%25255D_thumb.png?imgmax=800)
![clip_image005[16]](http://lh3.googleusercontent.com/-12riPotck8Y/VdYu1-k5YkI/AAAAAAAAREw/5Q0VZulSoss/clip_image005%25255B16%25255D_thumb.png?imgmax=800)
![clip_image006[16]](http://lh3.googleusercontent.com/-RFka6Lax6Uo/VdYu6A_b2EI/AAAAAAAARFA/A4Bf-7jcVcs/clip_image006%25255B16%25255D_thumb.png?imgmax=800)
![clip_image007[16]](http://lh3.googleusercontent.com/-rT42TujSLcg/VdYu-8T-rPI/AAAAAAAARFQ/ciVPOI7rLxI/clip_image007%25255B16%25255D_thumb.png?imgmax=800)
![clip_image008[16]](http://lh3.googleusercontent.com/-UXl0OP8ELlk/VdYvD2pe5PI/AAAAAAAARFg/57WU9bpLMWs/clip_image008%25255B16%25255D_thumb.png?imgmax=800)
![clip_image009[10]](http://lh3.googleusercontent.com/-rkCwzJAXkjY/VdYvHtlOI5I/AAAAAAAARFw/d8v0lBhiUEw/clip_image009%25255B10%25255D_thumb.png?imgmax=800)
![clip_image010[10]](http://lh3.googleusercontent.com/-k86U1B16iFM/VdYvLCaibNI/AAAAAAAARGA/ZpFHQMj8ysE/clip_image010%25255B10%25255D_thumb.png?imgmax=800)
![clip_image011[10]](http://lh3.googleusercontent.com/-_ywtt1T7iXA/VdYvQDekWkI/AAAAAAAARGQ/ekMQKbPkRvw/clip_image011%25255B10%25255D_thumb.png?imgmax=800)
![clip_image012[10]](http://lh3.googleusercontent.com/-WnSYxl4wcJQ/VdYvUBdYqKI/AAAAAAAARGg/svyAMxSfih8/clip_image012%25255B10%25255D_thumb.png?imgmax=800)
![clip_image013[8]](http://lh3.googleusercontent.com/-XYBxx8eiPNw/VdYvaQtFNKI/AAAAAAAARGw/MYy7SWKb_rw/clip_image013%25255B8%25255D_thumb.png?imgmax=800)
![clip_image014[8]](http://lh3.googleusercontent.com/-fE-Bou-7p0k/VdYve2i4AVI/AAAAAAAARHA/3Oi9fhkFXIM/clip_image014%25255B8%25255D_thumb.png?imgmax=800)
![clip_image015[8]](http://lh3.googleusercontent.com/-IWDQOZeADZM/VdYviimDQuI/AAAAAAAARHQ/M06O29i46gE/clip_image015%25255B8%25255D_thumb.png?imgmax=800)
![clip_image016[8]](http://lh3.googleusercontent.com/-C_kBbiR8EsM/VdYvmCKBC0I/AAAAAAAARHg/XXDJ1aRZeD8/clip_image016%25255B8%25255D_thumb.png?imgmax=800)
![clip_image017[8]](http://lh3.googleusercontent.com/-KBcrpzsqBE0/VdYvpcE6nGI/AAAAAAAARHw/g_GITksR_K8/clip_image017%25255B8%25255D_thumb.png?imgmax=800)
![clip_image018[8]](http://lh3.googleusercontent.com/-UjquZ6aPLb8/VdYvtJvGKrI/AAAAAAAARIA/hgFf-ybYbd0/clip_image018%25255B8%25255D_thumb.png?imgmax=800)
![clip_image001[20]](http://lh3.googleusercontent.com/-_2-TW0r6MQ0/VdYvxgJAaoI/AAAAAAAARIQ/Ivs_jXAZF-8/clip_image001%25255B20%25255D_thumb.png?imgmax=800)
![clip_image002[20]](http://lh3.googleusercontent.com/-7IiMqKkJ9Ok/VdYv2m7xufI/AAAAAAAARIg/PZVaQ-F-SNo/clip_image002%25255B20%25255D_thumb.png?imgmax=800)
![clip_image003[20]](http://lh3.googleusercontent.com/-Nzq2BO_Lw6M/VdYv6XqIuuI/AAAAAAAARIw/slgQeBTRoHE/clip_image003%25255B20%25255D_thumb.png?imgmax=800)
![clip_image004[20]](http://lh3.googleusercontent.com/-QjpvYftCPUI/VdYv9oP4MLI/AAAAAAAARJA/WibUlIZDQ2w/clip_image004%25255B20%25255D_thumb.png?imgmax=800)
![clip_image005[18]](http://lh3.googleusercontent.com/-ayenHxH2dPY/VdYwBhallTI/AAAAAAAARJQ/9ednPIvTK10/clip_image005%25255B18%25255D_thumb.png?imgmax=800)
![clip_image006[18]](http://lh3.googleusercontent.com/-8Ir_aw9XeOg/VdYwFLo2ZDI/AAAAAAAARJg/YcE87xZ-s8g/clip_image006%25255B18%25255D_thumb.png?imgmax=800)
![clip_image007[18]](http://lh3.googleusercontent.com/-3X4oKAJzwOk/VdYwJz6S1zI/AAAAAAAARJw/1_Aas9HmseQ/clip_image007%25255B18%25255D_thumb.png?imgmax=800)
![clip_image008[18]](http://lh3.googleusercontent.com/-ock6Aw6XQjo/VdYwNkSKwUI/AAAAAAAARKA/R6vTCITQSA4/clip_image008%25255B18%25255D_thumb.png?imgmax=800)
![clip_image009[12]](http://lh3.googleusercontent.com/-7rgUTq6ZRDQ/VdYwRmNYK6I/AAAAAAAARKQ/gPwPLgfrxuM/clip_image009%25255B12%25255D_thumb.png?imgmax=800)
![clip_image010[12]](http://lh3.googleusercontent.com/-mmQB2EMXEog/VdYwWfZktcI/AAAAAAAARKg/sVxElj3kQdE/clip_image010%25255B12%25255D_thumb.png?imgmax=800)
![clip_image011[12]](http://lh3.googleusercontent.com/--81XceSIe_c/VdYwapyrNwI/AAAAAAAARKw/nJjgRjRECRU/clip_image011%25255B12%25255D_thumb.png?imgmax=800)
![clip_image012[12]](http://lh3.googleusercontent.com/-knPkcwaE_fk/VdYwehFka4I/AAAAAAAARLA/4Pux7LjsL5o/clip_image012%25255B12%25255D_thumb.png?imgmax=800)
![clip_image014[10]](http://lh3.googleusercontent.com/-jGYdAI06rxg/VdYwmXHC-yI/AAAAAAAARLg/r3RHKKg0ufs/clip_image014%25255B10%25255D_thumb.png?imgmax=800)
![clip_image015[10]](http://lh3.googleusercontent.com/-d3BghjKBZZI/VdYwqCxJzmI/AAAAAAAARLw/zqvjhDw2l0s/clip_image015%25255B10%25255D_thumb.png?imgmax=800)
![clip_image016[10]](http://lh3.googleusercontent.com/-iBHqaOBtxII/VdYwumhIm9I/AAAAAAAARMA/S1mM3RW0A-M/clip_image016%25255B10%25255D_thumb.png?imgmax=800)
![clip_image017[10]](http://lh3.googleusercontent.com/-iXyllKI1qSc/VdYwyLyQ14I/AAAAAAAARMQ/46Xf9Doh1OM/clip_image017%25255B10%25255D_thumb.png?imgmax=800)
![clip_image018[10]](http://lh3.googleusercontent.com/-Q6V7rfHjIa4/VdYw1qzGdCI/AAAAAAAARMg/w8XCBhBTl-0/clip_image018%25255B10%25255D_thumb.png?imgmax=800)
![clip_image019[8]](http://lh3.googleusercontent.com/-r8RrrLaFN5M/VdYw5J_r7sI/AAAAAAAARMw/TRU_Y-J2-Mo/clip_image019%25255B8%25255D_thumb.png?imgmax=800)
![clip_image020[8]](http://lh3.googleusercontent.com/-GV6K3B-YSg4/VdYw8QbfjCI/AAAAAAAARNA/oZ8ghTMIJIw/clip_image020%25255B8%25255D_thumb.png?imgmax=800)
![clip_image021[8]](http://lh3.googleusercontent.com/-Ionvc8QzB9U/VdYw_zm5XlI/AAAAAAAARNQ/1VhsAxQz3tY/clip_image021%25255B8%25255D_thumb.png?imgmax=800)
![clip_image001[22]](http://lh3.googleusercontent.com/---Grd1bS7qs/VdYxDHsm3fI/AAAAAAAARNg/uTscrS8M4sI/clip_image001%25255B22%25255D_thumb.png?imgmax=800)
![clip_image002[22]](http://lh3.googleusercontent.com/-FoixBJkeY40/VdYxHItXTwI/AAAAAAAARN0/XwiQ6OH9lUg/clip_image002%25255B22%25255D_thumb.png?imgmax=800)
![clip_image003[22]](http://lh3.googleusercontent.com/-uZUGh6ELGeo/VdYxLUZG2PI/AAAAAAAAROE/dJhtJsAR0wQ/clip_image003%25255B22%25255D_thumb.png?imgmax=800)
![clip_image004[22]](http://lh3.googleusercontent.com/-FMxX7lOOUUY/VdYxPg7i0mI/AAAAAAAAROU/8V4mYBOdFfQ/clip_image004%25255B22%25255D_thumb.png?imgmax=800)
![clip_image005[20]](http://lh3.googleusercontent.com/-x-HJjsgZJEA/VdYxTU3kEkI/AAAAAAAAROk/ZL34fjKWIVo/clip_image005%25255B20%25255D_thumb.png?imgmax=800)
![clip_image006[20]](http://lh3.googleusercontent.com/-iKDAe0GfahY/VdYxYS6H3QI/AAAAAAAARO0/2dqvH-0itc4/clip_image006%25255B20%25255D_thumb.png?imgmax=800)
![clip_image007[20]](http://lh3.googleusercontent.com/-499mJtyVASU/VdYxeZTyojI/AAAAAAAARPE/3vx3xctVbaM/clip_image007%25255B20%25255D_thumb.png?imgmax=800)
![clip_image008[20]](http://lh3.googleusercontent.com/-lGvrY3uhL7c/VdYxhk3qfvI/AAAAAAAARPU/Zk-Unt1BIPY/clip_image008%25255B20%25255D_thumb.png?imgmax=800)
![clip_image009[14]](http://lh3.googleusercontent.com/-bjBLMPH3R1o/VdYxl5vfZYI/AAAAAAAARPk/6Kc0nUw9Q5c/clip_image009%25255B14%25255D_thumb.png?imgmax=800)
![clip_image010[14]](http://lh3.googleusercontent.com/-6aB0cJA7h60/VdYxpN8RomI/AAAAAAAARP0/s1m3JI9QIFM/clip_image010%25255B14%25255D_thumb.png?imgmax=800)
![clip_image011[14]](http://lh3.googleusercontent.com/-4joLGL8vaJ4/VdYxsjswwXI/AAAAAAAARQE/-Gr-StC06go/clip_image011%25255B14%25255D_thumb.png?imgmax=800)
![clip_image012[14]](http://lh3.googleusercontent.com/-fL6sGB14vK8/VdYxvy17c1I/AAAAAAAARQU/FRMpJxT4iuM/clip_image012%25255B14%25255D_thumb.png?imgmax=800)
![clip_image001[24]](http://lh3.googleusercontent.com/-ykz2SQr8sGY/VdYx5DWr7iI/AAAAAAAARQk/48zwtMZjjWA/clip_image001%25255B24%25255D_thumb.png?imgmax=800)
![clip_image002[24]](http://lh3.googleusercontent.com/-INonbG-RYTQ/VdYx-N8_fnI/AAAAAAAARQ0/aiTHcXo9-5g/clip_image002%25255B24%25255D_thumb.png?imgmax=800)
![clip_image003[24]](http://lh3.googleusercontent.com/-qa1xKJbYFRE/VdYyDtMvU6I/AAAAAAAARRE/_fH1iMkNB3I/clip_image003%25255B24%25255D_thumb.png?imgmax=800)
![clip_image004[24]](http://lh3.googleusercontent.com/-DJVgWbJd-sE/VdYyHK_2OYI/AAAAAAAARRU/i1d-M_ngRYI/clip_image004%25255B24%25255D_thumb.png?imgmax=800)
![clip_image005[22]](http://lh3.googleusercontent.com/-wyppRre3Ch0/VdYyM4c__OI/AAAAAAAARRk/UURRTjLtBCI/clip_image005%25255B22%25255D_thumb.png?imgmax=800)
![clip_image006[22]](http://lh3.googleusercontent.com/-dx4lvmcMDpM/VdYyRv6jHOI/AAAAAAAARR0/ny5Oubvg7B4/clip_image006%25255B22%25255D_thumb.png?imgmax=800)
![clip_image007[22]](http://lh3.googleusercontent.com/-FcKRl1xeL84/VdYyVUYuPQI/AAAAAAAARSI/IDplqROIUOo/clip_image007%25255B22%25255D_thumb.png?imgmax=800)
![clip_image008[22]](http://lh3.googleusercontent.com/-MojC5Pnp7Ec/VdYyZM7EIfI/AAAAAAAARSY/g_lRXSDxkNA/clip_image008%25255B22%25255D_thumb.png?imgmax=800)
![clip_image009[16]](http://lh3.googleusercontent.com/-vj4sDjWXbLY/VdYyddfdM-I/AAAAAAAARSo/6GaIXKkt9Q8/clip_image009%25255B16%25255D_thumb.png?imgmax=800)
![clip_image010[16]](http://lh3.googleusercontent.com/-TtlHi6ukXok/VdYyhPQa1UI/AAAAAAAARS4/miZAvJOXHFA/clip_image010%25255B16%25255D_thumb.png?imgmax=800)
![clip_image011[16]](http://lh3.googleusercontent.com/-L-__7dks3nI/VdYykgpQuII/AAAAAAAARTI/d-I2eDfCOlI/clip_image011%25255B16%25255D_thumb.png?imgmax=800)
![clip_image012[16]](http://lh3.googleusercontent.com/-Lv2ffy8xH88/VdYyoAd1OhI/AAAAAAAARTY/MjVrWM8LF1s/clip_image012%25255B16%25255D_thumb.png?imgmax=800)
![clip_image013[12]](http://lh3.googleusercontent.com/-X4IpS04KceY/VdYyrWhTOPI/AAAAAAAARTo/dVDkIzhtBGQ/clip_image013%25255B12%25255D_thumb.png?imgmax=800)
![clip_image014[12]](http://lh3.googleusercontent.com/-6iUvN0I4D84/VdYyvRtvC3I/AAAAAAAART4/1dYRAK4REbE/clip_image014%25255B12%25255D_thumb.png?imgmax=800)
![clip_image015[12]](http://lh3.googleusercontent.com/-qMyDa4Y7In0/VdYyy5Vx7BI/AAAAAAAARUI/_HH4uuiiLPA/clip_image015%25255B12%25255D_thumb.png?imgmax=800)
![clip_image016[12]](http://lh3.googleusercontent.com/-U1NYvxtHvy0/VdYy2NCq_4I/AAAAAAAARUY/yzsw5P091pY/clip_image016%25255B12%25255D_thumb.png?imgmax=800)
![clip_image007[1]](https://www.moh10ly.com/wp-content/uploads/2019/11/clip_image0071.png "clip_image007[1]")
![clip_image029[1]](https://www.moh10ly.com/wp-content/uploads/2019/11/clip_image0291.png "clip_image029[1]")
![clip_image030[1]](https://www.moh10ly.com/wp-content/uploads/2019/11/clip_image0301.png "clip_image030[1]")
![clip_image031[1]](https://www.moh10ly.com/wp-content/uploads/2019/11/clip_image0311.png "clip_image031[1]")
![clip_image029[2]](https://www.moh10ly.com/wp-content/uploads/2019/11/clip_image0292.png "clip_image029[2]")
![clip_image030[2]](https://www.moh10ly.com/wp-content/uploads/2019/11/clip_image0302.png "clip_image030[2]")
![clip_image031[2]](https://www.moh10ly.com/wp-content/uploads/2019/11/clip_image0312.png "clip_image031[2]")
![clip_image001[4]](http://lh3.googleusercontent.com/-tkkirEGtFK0/VcTCztUG_FI/AAAAAAAAQqg/p87NajYZcL4/s1600-h/clip_image001%25255B4%25255D%25255B2%25255D.png)
![clip_image002[4]](http://lh3.googleusercontent.com/-8qnRdArokQc/VcTC1Gps8wI/AAAAAAAAQqs/trt0STVFN90/s1600-h/clip_image002%25255B4%25255D%25255B2%25255D.png)
![clip_image003[4]](http://lh3.googleusercontent.com/-m-2u00_nLGs/VcTC2SKhXDI/AAAAAAAAQq8/3s9OXUk1npo/s1600-h/clip_image003%25255B4%25255D%25255B2%25255D.png)
![clip_image004[4]](http://lh3.googleusercontent.com/-BAM40H7TDqM/VcTC3Y9_ezI/AAAAAAAAQrM/XtwBgpjE0dA/s1600-h/clip_image004%25255B4%25255D%25255B2%25255D.png)
![clip_image005[4]](http://lh3.googleusercontent.com/-x9SooSP4njs/VcTC4xs109I/AAAAAAAAQrc/x7eGOX-nOGI/s1600-h/clip_image005%25255B4%25255D%25255B2%25255D.png)
![clip_image007[4]](http://lh3.googleusercontent.com/-nqJyeMtTbq0/VcTC664bsGI/AAAAAAAAQr8/16kubPj7JyE/s1600-h/clip_image007%25255B4%25255D%25255B2%25255D.png)
![clip_image008[4]](http://lh3.googleusercontent.com/-S3DPinIl_vU/VcTC8FjzqxI/AAAAAAAAQsQ/A5NVke_4Ljk/s1600-h/clip_image008%25255B4%25255D%25255B2%25255D.png)
![clip_image009[4]](http://lh3.googleusercontent.com/-JDO9q3PzSc8/VcTC9eUaJGI/AAAAAAAAQsc/Ub-UXWNmucI/s1600-h/clip_image009%25255B4%25255D%25255B2%25255D.png)
![clip_image010[4]](http://lh3.googleusercontent.com/-BtRFeRibyBc/VcTC-0Yr27I/AAAAAAAAQss/xciwuOhnuac/s1600-h/clip_image010%25255B4%25255D%25255B2%25255D.png)
![clip_image011[4]](http://lh3.googleusercontent.com/-AZlryqhxOkk/VcTC_zVufMI/AAAAAAAAQs8/VXQQhlfZ3ik/s1600-h/clip_image011%25255B4%25255D%25255B2%25255D.png)
![clip_image001[6]](http://lh3.googleusercontent.com/-t_MeWLx3CrQ/VcTDBF_DbLI/AAAAAAAAQtM/5_Uq36S2U40/s1600-h/clip_image001%25255B6%25255D%25255B2%25255D.png)
![clip_image002[6]](http://lh3.googleusercontent.com/-zocECvQh7ME/VcTDCDX-rjI/AAAAAAAAQtc/WqKpdHxtYY4/s1600-h/clip_image002%25255B6%25255D%25255B2%25255D.png)
![clip_image003[6]](http://lh3.googleusercontent.com/-6qs_vZQTzco/VcTDDZA-KqI/AAAAAAAAQtw/UoN-t3-i8HQ/s1600-h/clip_image003%25255B6%25255D%25255B2%25255D.png)
![clip_image004[6]](http://lh3.googleusercontent.com/-J9CN-nRD2vQ/VcTDH3mPPKI/AAAAAAAAQuA/2nBm48ARMSw/s1600-h/clip_image004%25255B6%25255D%25255B2%25255D.png)
![clip_image005[6]](http://lh3.googleusercontent.com/-p97LerOGvsw/VcTDJWj13mI/AAAAAAAAQuQ/1XFC_PmsYUA/s1600-h/clip_image005%25255B6%25255D%25255B2%25255D.png)
![clip_image006[4]](http://lh3.googleusercontent.com/-7wLY52_Dw7c/VcTDKpQygXI/AAAAAAAAQuc/LsTIUTUpSVA/s1600-h/clip_image006%25255B4%25255D%25255B2%25255D.png)
![clip_image007[6]](http://lh3.googleusercontent.com/-RQPFadMc1ss/VcTDL_m8UfI/AAAAAAAAQuw/XtXOh6d6LRA/s1600-h/clip_image007%25255B6%25255D%25255B2%25255D.png)
![clip_image008[6]](http://lh3.googleusercontent.com/-DOMFSF7xR90/VcTDM4E022I/AAAAAAAAQvA/TR96EY8FUd0/s1600-h/clip_image008%25255B6%25255D%25255B2%25255D.png)
![clip_image009[6]](http://lh3.googleusercontent.com/-esCxN6IaFDM/VcTDOCJG7KI/AAAAAAAAQvM/1D-3_Lb4-XY/s1600-h/clip_image009%25255B6%25255D%25255B2%25255D.png)
![clip_image010[6]](http://lh3.googleusercontent.com/-0TBV5c5L3FE/VcTDPX9y3LI/AAAAAAAAQvg/lX9Xn6a5fQ0/s1600-h/clip_image010%25255B6%25255D%25255B2%25255D.png)
![clip_image011[6]](http://lh3.googleusercontent.com/-zew_4ZsSA8c/VcTDQj6YSjI/AAAAAAAAQvw/2wuNd_IPAk0/s1600-h/clip_image011%25255B6%25255D%25255B2%25255D.png)
