Maybe you have been in this situation before, trying to delete an Exchange database after moving all of its mailboxes, arbitration or archives to another server or database but it didn’t work and said that there is still something in the database?
Now I am in a similar situation however I checked nothing in the database as you can see in the below screenshot.
I have noticed that these issues could happen when an Exchange server gets broken or forcefully deleted from AD without properlty uninstalling it. some traces of system mailboxes might remain there with database attributes pointing to the database.
– You will have to run this script from Exchange Server.
– An account that can connect to Active Directory with at least read permission and Exchange admin read role.
The script will utilize Active Directory and Exchange to get the Database’s distinguished name and scan any AD User Object that has this DB’s DN and post it to you as an output.
As you can see in the below screenshot, I got mostly health mailboxes which should not really be a problme in case you’re deleting database, however I got one system mailbox that is still there and pointing to this Database however, I already have scanned the database for any arbitration mailboxes but Exchange CMDlet showed none.
Now that I know which user it is, the solution for me to be able to remove/delete this database which I no longer need is to delete this AD user object since its no longer in use by Exchange.
You won’t be able to get this mailbox through get-mailbox cmdlet because its not an active mailbox. however you will find it in AD.
So I deleted the mailbox mentioned below and next I am going to try and delete the mailbox database in question.
Result
Here’s the result after deleting the user in Question.
After the disastrous exploit that was found in Microsoft Exchange Servers lots of corporations started immediately patching their servers with the latest Cumulative update and Security patches. The question is would those patches be enough if the server is already hacked or have a backdoor installed already?
What are those 0-day exploits ?
The vulnerabilities recently being exploited were CVE-2021-26855, CVE-2021-26857, CVE-2021-26858, and CVE-2021-27065 which are part of alleged “State-sponsored Chinese group” according to Microsoft.
Let’s get into details of those exploits one by one:
CVE-2021-26855 is a server-side request forgery (SSRF) vulnerability in Exchange which allowed the attacker to send arbitrary HTTP requests and authenticate as the Exchange server.
CVE-2021-26857 is an insecure deserialization vulnerability in the Unified Messaging service. Insecure deserialization is where untrusted user-controllable data is deserialized by a program. Exploiting this vulnerability gave HAFNIUM the ability to run code as SYSTEM on the Exchange server. This requires administrator permission or another vulnerability to exploit.
CVE-2021-26858 is a post-authentication arbitrary file write vulnerability in Exchange. If HAFNIUM could authenticate with the Exchange server then they could use this vulnerability to write a file to any path on the server. They could authenticate by exploiting the CVE-2021-26855 SSRF vulnerability or by compromising a legitimate admin’s credentials.
CVE-2021-27065 is a post-authentication arbitrary file write vulnerability in Exchange. If HAFNIUM could authenticate with the Exchange server then they could use this vulnerability to write a file to any path on the server. They could authenticate by exploiting the CVE-2021-26855 SSRF vulnerability or by compromising a legitimate admin’s credentials.
How to proceed ?
Microsoft released couple of tools that could diagnose your servers and check if you already have been infected with a backdoor or any of these nasty malware and also remove those infected files or clean them and ask you for a restart if it’s required.
Tools:
MSERT (Microsoft Safety Scanner) detects web shells, Download here .
Health Checker (Scans your server for any vulnerabilities and whether you have updated Server CU and installed patches). Download here
Exchange WebShell Detection (A simple PowerShell that is fast and checks if your IIS or Exchange directory has been exploited). Download here
Microsoft very recently created a mitigation tool for Exchange on-premises that would rewrite url for the infected servers and recover the files that were changed. You can download the tools from this github link.
Copy the Test-ProxyLogon code into Notepad
Save As “Test-ProxyLogon.ps1” with the quotes in your C:\Temp folder
Run in Exchange Management Shell: .\Test-ProxyLogon.ps1 -OutPath C:\Temp
Scan Result
Scan result should show you the following if your servers has been exploited already.
This will remove the infections and asks for a restart.
CVE-2020-0688 | Microsoft Exchange Validation Key Remote Code Execution Vulnerability
Security Vulnerability
Date of Publishing: February/11/2020
Microsoft has announced a vulnerability has been found in all Exchange Server 2010 through 2019 versions, The vulnerability allows an attack to send a specially crafted request to the affected server in order to exploit it.
When could this happen?
A remote code execution vulnerability exists in Microsoft Exchange Server when the server fails to properly create unique keys at install time.
Knowledge of a the validation key allows an authenticated user with a mailbox to pass arbitrary objects to be deserialized by the web application, which runs as SYSTEM.
The security update addresses the vulnerability by correcting how Microsoft Exchange creates the keys during install.
Affected Versions:
Microsoft Exchange Server 2010 Service Pack 3 Update Rollup 30
Microsoft Exchange Server 2013 Cumulative Update 23
Microsoft Exchange Server 2016 Cumulative Update 14
Microsoft Exchange Server 2016 Cumulative Update 15
Microsoft Exchange Server 2019 Cumulative Update 3
Microsoft Exchange Server 2019 Cumulative Update 4
Solution:
Until now Microsoft has not provided any solution or work around to cover this vulnerability.
Mitigations
Microsoft has not identified any mitigating factors for this vulnerability.
Workarounds
Microsoft has not identified any workarounds for this vulnerability.
In office 365 when you’re working on Exchange 2010,2013, 2016 or 2019 in a hybrid environment things might look easy but in a big enterprises where Internet security is something being taken into account very seriously. It might cause many issues that you don’t expect at all.
One of my clients whom I was doing Exchange Migration for had an issue with the Migration. The error was as follows:
Error occurs after Office 365 Exchange online connects to Exchange on-premises 2010 mailbox server
RPR05DG049-db131′, Mailbox server ‘DB3PR05MB0778.eurprd05.prod.outlook.com’
Version 15.1 (Build 466.0).RequestExpiryTimestamp : 03.04.2116 07:42:38
ObjectState : New
Troubleshooting:
To troubleshoot issues, You need to put so many things into account! The architecture of the infrastructure of where you are doing the project is very important and the need of knowing how things are working matters.
Things that could always come in mind and handy are what you will need to start your troubleshooting:
To troubleshoot the MRs, You need to know what kind of error you’re getting and to see this you can use the following powershell after you connect to Office 365 powershell.
The resultant report will reveal the error and shows you where is the exact culprit.
– Disk Latency
– Firewall Configuration (IPS/IDS)
From Exchange 2016 to 2019 or 2013 to 2016 The transient error might be related to MRSProxy or at least this is the case with me 90% of the time. To resolve this issue you will need to change the MRSProxy values on the target server and depending on the error might also be the Source server too.
SOLUTION:
===========
1. Some instability was detected in communications as well as saturation by the size of the link.
2. The procedure to increase the timeout for the service through the file MRSProxy
This guide will show you how to installing order to Install “Only” Trend Micro 11.0 on Exchange 2013 server
You will have to make sure that before you install Trend Micro you have enough resources on the mail servers or Edge servers depending on where you are intending to install it.
Prerequisites:
You will need to install Windows IIS CGI role.
Net Framework 3.5
Trend Micro Setup.
If you did not install CGI you will get the following error, so you must install it
To install it you will need to go to Add Roles and then choose and install it.
If Net Framework 3.5 is not install the setup won’t proceed unless you do so and you will get the following error:
To install Net Framework 3.5 , you can use the wizard or you can use the Powershell but you’ll need to attach Windows Server ISO File to the VM or the physical machine.
Setup will restart from the beginning
NetFrame work fails from the Server Manager
Instead, I imported the Windows 2012 r2 server ISO into the VM and ran the powershell command line
Restarted the Trend Micro Setup and the setup is working
I already have copied the setup files on my mailbox servers, in my scenario I have 2 mail box servers which I am going to install it on.
I will launch the setup and go through the following wizard
As I mentioned earlier, I am planning to install it on Exchange 2013 Mailbox servers, so I will go ahead and choose Mailbox servers
I will click Browse and Add exchange servers and as in the following snapshot it’ll show me total server count
Next I will type the Exchange Admin account which I used to setup Exchange with and login to the admin Center which is also a local admin.
This is set by default so you will need to leave it as it is.
You can keep the following default settings or change the port in case it’s already used or enable SSL.
In my case I will enable SSL as well as it’s more preferable for security purposes.
Trend micro setup will check if there’s any previous instance on the target Mailbox server in order to check if it’s an upgrade or a fresh install.
I have no proxy so I will proceed without it.
I’m planning to ignore this now and register later, so you can provide the key if you already have it and want to register.
When you continue without activating the product you will get the following warning.
Depending on if you wanna be useful or not, you can just to participate with this program or just ignore it.
In case you would like to direct or send all incoming spam messages to the user to take the decision him/her self you can choose to integrate with Outlook junk e-mail or integrate with End user’s quarantine. In this case incoming infected or suspicious mails will be delivered to the user’s Quarantine but can be restored from/with trend micro.
Trend Micro have also a control manager for centralized management, so if you have it you can configure it and manage all those scanmail from one location. If not then just click next
Click browse and choose your domain in order to select the domain admin groups to manage the trend micro scan mail application.
All server details and configuration is going to be listed in the next snapshot.
And now installation should start.
The credentials to login might be standard but you could also try your domain admin which you have assigned during the setup to login to the portal.
Any configuration that you do on the Mailbox server 1, you will have to re-do it on Server 2 since this is not centralized management.
So first thing I’ll do is update the product to the latest version.
After selecting the components to update click on Update and wait for the process to finish.
After setting and configuring couple of rules and restarting Exchange transport service on each server . I was able to test It and see that it works as in the following snapshot.
In order to export mails from Exchange 2003 (should not exceed 2 GB) you will have to copy Administrator user into another user “admin” and give that user the rights to access all other mailboxes.
You will have to navigate to the Mailbox store
Right click the mailbox store and click on Properties
Go to Security tab and add the new user (Admin) and give it full control as below
Apply, then sign out of the windows session to the Exchange machine and use the newly added domain admin to login and then open the Exmerge application
Select the second step (Extract or Import)
Select step1
Select the Exchange name and the DC (They should be set automatically)
Select the users that you want to be exported (shouldn’t exceed 2 GB).
Select the local language
Select the destination folder (In my case I mapped a network drive)
Save settings for later use if you want or just click Next.
Once done, the mailbox will be exported.
Importing into Exchange 2013
In exchange 2013 Open the EMS as administrator
Before you start, you should move all the PST files into a shared folder in the network and add the “Exchange Trusted Subsystem” user to its permission.
The same user should be added to the security tab
Providing import and export permission on Exchange 2013
In order to import the PST files to Exchange 2013 users you will have first to assign the Exchange Admin account the capability of importing these PST files then sign out from the EAC portal and back in
To do so you will have to go to EAC then go to Permissions and double click on the Recipient Management
Click Add and select the Mailbox Import Export and click Add then OK
I will add members to this role group
After signing in back to the EAC with the administrator I got the Import PST options.
Step by Step Installing Exchange server 2013 from scratch (Part 1)
In this part, I will be demonstrate how to Install exchange 2013 and prepare new Databases along with preparing the servers for high availability (DAG).
Prerequisites:
– Two Microsoft Windows 2012 R2 servers with 16 GB ram and 200GB disk divided unto two partitions.
– Two NIC, one for MAPI and one for replication.
– Exchange 2013 CU8 setup to directly go to the latest available update.
Installing Prerequisites on all exchange servers
Launch Powershell as administrator and copy then paste the following.
You should download and install the following software prerequisites as per Microsoft’s TechNet article regarding the installation. the software is available the link I posted earlier above or through this link Exchange_Prerequesties
After running all the prerequisites , we can start installation of Exchange 2013
Here I am going to change Exchange’s default installation path and place it on a different partition to avoid any data loss in case of Windows server crash or booting issues.
Now we install the second Exchange server, that will hold the same roles on it as the first one (Mailbox and CAS).
The steps are going to be exactly the same except that you won’t have to prepare the schema or AD since it’s already prepared.
Installation has finished for both servers
Creating DATABASES:
NOTE:
It’s better to mount the database upon creation and not restart the IS instantly after that.
Now it’s time to create new Databases and replace them with the default ones that come with the Installation
First we’ll have to start off by creating our targeted databases which we want to use them. Note that for the standard version of Exchange 2013 you can only create up to 5 databases per mailbox server.
In order to demonstrate all benefits of Exchange 2013 and its features including DAG. I will create 2 databases. One database on each server.
The first database will be called DB1SRV1
As soon as we have created the Database, we faced the following error with event ID 106
Then another warnıng from MSExchangeFastSearch wıth event id 1006
This indicates that a database should not be mounted upon creation, you should untick the mount DB option when you create one.
After waiting a bit the following logs should appear and show a healthy indexing start.
Once the DB has been created, Exchange AC will require that you restart the IS (Information store Service) in order for replication to happen without an issue.
Database is showing healthy and no issues so far.
Now we’ll create a new DB on the second server without ticking the mount DB option.
Microsoft Exchange Server Locator Service failed to find active server for database ‘de5f3051-c202-4976-b8e4-65bbbe0c2395’. Error: The database with ID de5f3051-c202-4976-b8e4-65bbbe0c2395 couldn’t be found.
The same exact errors came after creating the Database without mounting it.
Now let’s restart the IS service and mount our database then see what happens..
Upon restarting the service, we get the following error which is related to the MS Exchange replication service . It noticed that the database that we have created has never been mounted in order to start the indexing.
Let’s mount the database and see the changes
Mounting the database have got the AM to report successfully and after couple of seconds the MSExchangeFastSearch will check out if the database have any indexing files.
No indexing state have been found and so the FastSearch service will give you a 1013 warning report. This is a good warning because it reports that the service is working properly and that it will create the indexing folder after couple of minutes as we’ll see later.
It takes approximately 3-5 minutes for the database to start the indexing.
Now on the EAC, the DB should report healthy. Let’s see
Removing Default databases
First step before deleting the default databases is to move any system mailboxes or arbitrary mailboxes in them to the newly created databases…
Paul Cunningham wrote a great article on how to do this using Powershell … in the following link
The warning above is apparently due to Exchange permission on AD. It has been described in detail on how to solve this warning by Nuno Mota in the following Link.
Stack trace: at Microsoft.Exchange.Clients.Common.UserAgent.HasString(String str)
at Microsoft.Exchange.Clients.Common.UserAgent.get_Layout()
at Microsoft.Exchange.Clients.Common.UserAgent.get_LayoutString()
at ASP.auth_logon_aspx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)
at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
at System.Web.UI.Page.Render(HtmlTextWriter writer)
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
Custom event details:
OWA PAGE
Solution:
On Exchange servers, Make sure that Exchange servers are not members of Organization Management group and if they are then remove them and run this cmdlet anyway on all Exchange Servers then restart the Servers.