Friday, October 25, 2019

Exchange SpamTitan Dynamic Recipient Verification With Edge Server

You may be reading the title of this post and wonder "why would you have an Edge Server and a spam filter?". Funny story: Well, not so funny. Back during planning of my Exchange environment, it was suggested by a Lotus Notes admin (who knows nothing about Exchange) that we should have an Edge. I (the Exchange admin) said "no, that's overkill". Well, my advice wasn't taken and we installed the Edge. The "funny" part: 3 years later I was asked "can we remove the Edge?" Come on! Of course not, it's there, it's staying put!

SpamTitan will do Recipient Verification to automatically bounce incoming messages addressed to invalid recipients, which can cut down on the amount of spam quite a bit; it also uses Recipient Verification for licensing. 
That last piece is important if you only pay for say, 1000 users, but the appliance sees 1,500 recipients.

Dynamic Recipient Verification (DRV) is the easiest to manage - there are other options like LDAP and manual import, but no one wants to mess with that. 
To use DRV, we need to enable Recipient Filtering on our Exchange environment. SpamTitan will then check with Exchange to verify if a recipient is valid or not, and either pass it through or bounce it.

SpamTitan has the following guide to enable DRV, but it only applies to Mailbox Servers, it doesn't cover if you're running an Edge Server.


The guide requires your Mailbox Servers to be internet-facing and involves allowing anonymous authentication on the Default Hub Transport connector on port 2525.
Edge Servers don't have a Default Hub Transport Connector, so there's no way to allow connections on port 2525.

I'll show you how to enable Dynamic Recipient Verification when you have an Edge in play.

First, enable Recipient Filtering:

On your Edge Server, in the Exchange Management Shell (EMS), run the following cmdlet to check our Recipient Filtering settings:


This should show the default settings, if you've never enabled any recipient filtering. The main thing to look for will be:

Name                                      : RecipientFilterConfig
RecipientValidationEnabled  : False
Enabled                                  : True

We'll enable Recipient Filtering with the following cmdlet:

Set-RecipientFilterConfig -RecipientValidationEnabled $true

Now, run the first cmdlet, and you'll see it enabled and ready:

Name                                    : RecipientFilterConfig
RecipientValidationEnabled  : True
Enabled                                 : True

Next, we'll enable our Accepted Domain(s) for Address Book recipient lookup:

Run the following cmdlet to check if Address Book is enabled:

Get-AcceptedDomain | FL Name,AddressBookEnabled

If it isn't enabled, run the following for each Authoritative Accepted Domain:

Set-AcceptedDomain "name of accepted domain" -AddressBookEnabled $true
**Note** Change "name of accepted domain" to your Accepted Domain.

**Note** You only want to set this on Authoritative Domains, not Relay domains.

Next, configure Dynamic Recipient Verification on the SpamTitan:

Browse to Setup > Mail Relay > Domains.  

Edit your domain(s) by clicking the "Pencil" icon, and select Dynamic Recipient Verification from the drop-down menu. 

Enter your Edge Server IP or host name (this will usually match your Destination Server):

SpamTitan Edit Domain

Click Save, and you'll see that DRV is enabled for your domains:

SpamTitan DRV Enabled

And lastly, your license will reflect the correct recipient count:

SpamTitan License Count

**Note** When Recipient Verification is not enabled, you'll get a warning that you have exceeded your recipient count. If that happens, you can reset the count (up to 3 times) and then wait a day or so and DRV will show the correct count.

Now you can test by sending an external message to a bunk internal email address and watch it bounce:

SpamTitan Bounce

The response from the remote server was:

550 5.1.1 <stace@exchangeitup.com>: Recipient address rejected: undeliverable address: host[] said: 550 5.1.10 RESOLVER.ADR.RecipientNotFound; Recipient not found by SMTP address lookup (in reply to RCPT TO command)

As you can, we got a bounce from SpamTitan, and an explanation from our Edge server saying the address is invalid.

**Note** You can test by using telnet, but a lot of organizations don't allow telnet to port 25, so we're using just a regular email test.

Now w're all set! License count looks good, and dirty spammers will be bounced when they try to guess recipients.

Saturday, August 24, 2019

Exchange Veeam Backup Error 0x800401fd: Unable To Truncate Exchange Logs

We use Veeam to back up our Exchange 2016 environment. It generally works ok; if I had my way (and upper management understood Exchange) I'd run backup-less since we have a DAG and all...but whatever.

Every so often Veeam throws errors, mostly dealing with VSS issues; see my old posts here and here.

Now, I just got a new one, where Veeam finishes the backup job, but couldn't truncate the Exchange Transaction logs, causing the log volumes to get pretty big.

Veeam Error:

Unable to truncate Microsoft Exchange transaction logs. Details: Failed to call RPC function 'Vss.FinishSnapshot': Error code: 0x800401fd. Failed to invoke func [FinishSnapshot]: Object is not connected to server.

Exchange Error:

On the Exchange DAG node that Veeam connects to, there's also the following Application Event 2034:

Log Name:      Application
 Source:        MSExchangeRepl
 Date:          8/16/2019 8:51:56 PM
 Event ID:      2034
 Task Category: Exchange VSS Writer
 Level:         Error
 Keywords:      Classic
 User:          N/A
 Computer:      MBX1.exchangeitup.com
 The Microsoft Exchange Replication service VSS Writer (Instance c2f51bd6-b7b3-49e1-8e12-4ad329955268) failed with error FFFFFFFD when processing the backup completion event.

This means our VSS Writer on the Exchange server is malfunctioning.

To check for VSS errors, fire up the CMD on the Exchange server, and run:

vssadmin list writers

You'll most likely get the following:

Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {2e1494ca-6e06-47a8-af0e-f77213bfecaf}
State: [12] Failed
Last error: Retryable error

The Fix:

Luckily the fix is easy!

A lot of forums will say to restart the Exchange Store service. DO NOT DO THIS! That results in database dismounts, and client connectivity loss.

To "reset" the VSS writer properly, restart the Exchange Replication Service (You'll notice the above event references the Repl service). This will not cause any downtime, as its for the database replication across servers.

Open the EMS (Exchange Management Shell) on the Exchange server, and run the following cmdlet to stop the Replication Service:

Stop-Service MSExchangeRepl

Now start it with the following:

Start-Service MSExchangeRepl

Now, let's check the VSS Writers again, and you'll see the error cleared:

Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {12c870b7-b0b2-418c-88c0-3564025eb01c}
State: [1] Stable
Last error: No error

Re-run your backup, or let it run on the next schedule, and you should get a successful backup, with logs truncated!

Saturday, August 17, 2019

Exchange 2016 EMS Error: Could Not Load Microsoft.Exchange.CabUtility.dll

A few months ago, I upgraded my Exchange 2016 Servers to CU12, which included a Management Tools box that I run so other "non-Exchange" admins don't connect directly my Exchange machines.

The CU12 install was successful, but I just noticed that the EMS (Exchange Management Shell) on the Management Tools server threw an error when remoting to the Mailbox Servers. It was complaining that it couldn't find some required files; namely "Microsoft.Exchange.CabUtility.dll".

The Error:

Exception calling "EnsureTargetTypesLoaded" with "2" argument(s): "Could not load file or assembly
 'Microsoft.Exchange.CabUtility.dll' or one of its dependencies. The specified module could not be found."
 At C:\Program Files\Microsoft\Exchange Server\V15\bin\RemoteExchange.ps1:79 char:1
 + $typeLoadResult = [Microsoft.Exchange.Configuration.Tasks.CmdletAssemblyHelper]: ...

    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
     + FullyQualifiedErrorId : FileNotFoundException

 C:\Program Files\Microsoft\Exchange Server\V15\bin\RemoteExchange.ps1 : The Exchange types file wasn't loaded because not all of the required files could be found.
 At line:1 char:1
 + . 'C:\Program Files\Microsoft\Exchange Server\V15\bin\RemoteExchange.ps1'; Conne …

After some digging around on the server, I noticed that Visual C++ Redistributable for Visual Studio 2012 wasn't installed - Exchange 2016 CU12 requires Visual Studio 2012 and 2013.

What's even stranger: Visual Studio 2012 was installed prior to installing Visual Studio 2013 during the CU12 install...does 2013 overwrite 2012? I dunno, but this very same thing (Visual Studio 2012 missing) happened on my Edge server, which is how I knew to look there.

Again, this was strange since the upgrade finished with no errors. You'd think if something was missing, the install would bomb out...shrug.

The Fix:

Close the EMS.

Install Visual C++ Redistributable for Visual Studio 2012 Update 4 from here.

Fire up the EMS...no more errors!

Sunday, June 23, 2019

Exchange Inbox Rule To Move Email To Folder Sent To Alias

Here's a scenario I've seen quite often:

You have a Shared Mailbox called Human Resources (hr@exchangeitup.com), and you've added an alias of training@exchangeitup.com.

One of the user's with Full Access to the HR Mailbox, wants to create an inbox rule that will move messages sent to the training address to a Training folder - but this won't work.

When messages are sent to an alias, Exchange Directory Services will "translate" the alias as the primary email address; Training = Human Resources, and you can see it action by typing the address in the "To" field in Outlook and watch it change to the primary name. Therefore, inbox rules won't work for sorting/moving those emails to the alias.

In order to get around that behavior, we'll need to create a Mail Contact and a Transport Rule:

1. Remove the alias from the Shared Mailbox by running the following cmdlet in the Exchange Management Shell (EMS):

Set-Mailbox "Human Resources" -EmailAddresses @{remove="training@exchangeitup.com"}

Give it a few for Exchange/AD to update.

2. Create a Mail Contact for "Training" by running the following cmdlet:

New-MailContact -Name "Training" -ExternalEmailAddress training@exchangeitup.com -OrganizationalUnit “Contacts” 

**Note** This is re-using the secondary email address taken away from shared mailbox in step 1. Change the Name, External Email Address, OU match your environment.

3. Create a Transport Rule to redirect all messages sent to the Training Contact over to the HR Shared Mailbox:

 In the Exchange Admin Center (EAC), navigate to mail flow > rules.

Create a new Rule with the following settings:

Name: Training Forward

*Apply this rule if...
The recipient is..."Training"

*Do the following...
Redirect the message to..."Human Resources"

Or  you can use the EMS to run the cmdlet:

New-TransportRule, parameters -SentTo ("Training") -RedirectMessageTo ("Human Resources") -Name "Training Forward" -StopRuleProcessing "False" -Mode "Enforce" -Comments "" -RuleErrorAction "Ignore" -SenderAddressLocation "Header"

4. Create an inbox rule for the HR Shared Mailbox to move emails sent to the Training Contact to another folder in the Shared Mailbox:

In OWA, click your person icon in the upper-right corner, then select Open Another Mailbox.

Search for and open the Shared Mailbox (Human Resources in this example).

Then click the Settings Gear in the upper-right and go to Options > Inbox and sweep rules > create a rule using the following settings

It was sent to..."Training"
Do all of the following:
Move the message to folder..."Training Folder"

Now test it out! Send a message to training@exchangeitup.com and verify that the message was moved directly to the Training Folder.

Saturday, May 25, 2019

Exchange Forced TLS With SpamTitan

The growing trend nowadays, especially when sending messages to banks, is to use Forced TLS (also called Mandatory TLS). On top of TLS, most banks are now also requiring that the sending host use a public certificate to verify "you are who you say you are" - this is generally done with SPF records, but security requirements are getting more stringent.

If you're sending directly from Exchange and not using a spam filter (which probably isn't very common these days) you would just set up a Partner Send Connector and be done - except for assigning a cert to the connector, which is pretty tricky.
Since most organizations do use some sort of outgoing appliance (like SpamTitan), this post will walk you through setting up Forced TLS from Exchange on out through that gateway.

In my environment, we receive messages through a SpamTitan filter (which is awesome), but we send out through a Totemo encryption gateway...ever heard of it? Didn't think so. My short review on that product: It's garbage.
I was forced by company higher-ups to use it, going against my recommendations because it's not good practice to send and receive through different hosts/IPs - Totemo doesn't receive mail and it doesn't work very well...it runs on Java, which should tell you all you need to know. But it uses the "encrypt" button in Outlook to make users feel better, which adds complexity and results in help tickets...daily.
So, I decided to use the SpamTitan for outgoing TLS connections and it works perfectly!

Exchange Send Connector

We'll need to create a special Send Connector on Exchange to use the SpamTitan as a Smart Host.

**Note** Even if you're already using something like SpamTitan as your smart host on your current Send Connectors, I find it cleaner to create a dedicated connector for TLS - this way you can manage the domains requiring TLS without affecting your regular mailflow.

In the EAC (Exchange Admin Center) navigate to Mail Flow > Send Connectors. Click the "+" to create a new one:

New Send Connector
Give the Send Connector a name like "Google - TLS" and choose "Custom":

Custom Send Conenctor

**Note** We don't need a partner connector in this case, because mail will flow through our smart host not directly to our recipients.

Choose "Route mail through smart hosts", click the "+" and add your SpamTitan public IP address. Then click Next:

Send Connector Smart Host

There will be no authentication, so leave "None" selected and click Next:

Send Connector No Auth

In the Address space, click the "+" and enter the domains that you will sending over TLS to. If you want to include subdomains (which you always should), put an asterisk in front of the domain name. For instance to send to all of google/gmail, you'll enter *.google.com. Click Next when you've entered the domains:

Send Connector Address Space

In the Source server, click the "+" and choose your Edge Server(s) if you have them, otherwise choose your internet-facing Mailbox Server(s) and click Finish:

Send Connector Source Server

Your Send Connector is now created. Double-click it and change logging to "Verbose" and click Save. This will allow us to diagnose any issues later on by checking protocol logs, should TLS not work correctly:

Send Connector Logging

SpamTitan SSL Settings

On the SpamTitan WebUI, we need generate a CSR (Certificate Signing Request) so we can assign a public cert to send and receive with.

On your SpamTitan appliance, navigate to Settings > SSL. In the "Generate CSR" section, fill out the company info according to values below:

Common Name: The FQDN of your SpamTitan host
Organization: Your company name
OU: Optional
City/State/Country: The city, state and country your company is based in.

Then, hit "Run" under the "Generate Certificate Signing Request (CSR)" section.
SpamTitan CSR

Once the cert request is generated, copy the entire code block including "BEGIN CERTIFICATE REQUEST--- to ---END CERTIFICATE REQUEST---" and give that to whoever requests your certs from a Public CA.

When you get your cert generated and sent back to you in .pem format, import it into the SpamTitan under the "Import Certificates" section.

Under "Import Certificate from PEM" click "Choose File". Select your cert.pem file, and click "Import":

SpamTitan Cert import

You can view your imported cert under "Installed Signed Certificates" by clicking the magnifying glass icon:

SpamTitan Cert Properties

You'll then be presented with the cert details to make sure it's correct:

SpamTitan Cert

SpamTitan TLS Settings

Now that we have the cert in place and SSL enabled, we need to enable and configure TLS on the SpamTitan.

Navigate to Settings > TLS and click "Enable" for TLS. Select your new public cert, and enable logging:

SpamTitan TLS

**Note** This will also enable TLS for receiving and assign that cert for outside senders who need to verify your mail system (some banks require certs on both sending and receiving ends).

Next, navigate to System Setup > Mail Relay > Outbound.

Under "Trusted Networks" we'll add our Exchange external IP address(es). In my case it's my Edge server, but this could also be your internet-facing Mailbox Server(s):

SpamTitan Trusted Network

Under "TLS Encryption" we'll enable TLS, and in the drop-down choose "Use TLS for specified domains and Opportunistic TLS for all other connections". We do this because we don't want to force TLS on hosts that don't support it, which will drop connections:

SpamTitan TLS Encryption

Next, we'll add our recipient domains (these will match the domains we added in our Exchange Send Connector) with the following settings:

Domain: the top-level domain name
Include sub-domains: Yes
Policy: Mandatory TLS Encryption (aka Forced)
Allowed SSL/TLS Protocols: TLSv1/TLSv1.2 (these are most commonly supported)
Comment: Optional

SpamTitan TLS Domain

The domain will now be listed:

SpamTitan TLS Domain Listed
Under "Send Client Certificate" choose "enable" and verify that your public cert is selected and click "Save":

SpamTitan TLS Cert

Now, send a message to your gmail address (if you have one) or send a message to your external domain contact and verify that TLS was enforced by checking the receiving email header. You'll see the TLS version used to send the message:

TLS Verification

All done! Now you'll be sending secure messages!

Saturday, May 11, 2019

Exchange Transport Rule Block Attachments While Allowing Specific Ones

In my Exchange environment we have an invoicing mailbox that auto-forwards to an external document processing email address. This mailbox can only accept PDF or TIFF attachments, since that's what the doc processing service will accept. So, I needed a way to only allow those two attachment types, while blocking all others.

Articles on the Web are spotty on how to accomplish this...most are all or nothing. MS TechNet has an article on blocking all attachments, but we need to allow the ones mentioned above to come through. In this post, I'll expand on that.

We'll be using the EAC (Exchange Admin Center) to create our Transport Rule (aka Mail Flow Rules) since it's a little bit easier to see all of our options.

In the EAC, navigate to Mail Flow > Rules.
Click the "+" and select "Create a new rule...":

EAC Create New Rule

In the rule creator, give it a name.

Under "*Apply this rule if..." select "the recipient is..." and then search for and select your recipient(s).

Click the "more options" link and then click "add condition".

Drop-down to "any attachment" then mouse over to "is greater than or equal to..."

Input "1" in the field - for 1KB.

**Note** The EAC will only accept 1KB as the lowest value, which will block all attachments in my experience. If you need to go lower, like 0KB, you'll need to use the Shell to set it later.

Under "*Do the following..." drop-down to "Reject the message with the explanation..." and enter your outgoing bounce reason. In my case I specify that only PDF and TIFF are allowed.

Under "Except if..." drop-down to "Any attachment's file extension matches..." and enter "tiff" and "pdf".

**Note** Don't enter a dot in front of the file extensions, as it will throw an error; only enter the extension name.

Set your priority and leave "enforced" selected.

Click "Save"

The results should look like so:

EAC Attachment Rule Settings

Basically, what we did is block all attachments, but since we set an exception, we allowed those two specific ones through.
This might seem counterintuitive at first, but think if you did it the other way around and had to manually block every file type...that would take forever!

**Note** In the above example I set the rule for one mailbox, but if you need to allow only these certain attachments to all mailboxes, under "Apply this rule if..." you would choose "the recipient is... located inside the organization". You can also set it to "a member of this group" if you have groups set up for different settings.

Now, test your rule by sending a blocked attachment like a JPEG, to the mailbox and then send an allowed attachment like a PDF.

The blocked attachment should generate a 5.7.1 reply like so:

Delivery has failed to these recipients or groups:

invoices@exchangeitup.com (invoices@exchangeitup.com)

This message was rejected because only PDF or TIFF attachment types are allowed.

Your message wasn't delivered because the email admin for the organization 'exchangeitup.com' created an email rule restriction. Please contact the email admin for that organization and ask them to remove or update the rule restriction.
For more information about this error, see DSN code 5.7.1 in Exchange Online - Office 365.

As you can see, the sender will get the 571 bounce, and it includes my custom outgoing message under the recipient email address: "This message was rejected because only PDF or TIFF attachment types are allowed."

Happy attachment blocking!

Saturday, May 4, 2019

Outlook Teams Add-in Overrides Skype Meeting Add-In

My company uses Teams for meetings internally, but many of our clients use SFB on-premises, so we need to be able to schedule meetings in both modalities depending on the attendees.

The problem is: once Teams is installed, the Teams Outlook add-in takes precedence over the SFB add-in - the SFB add-in actually gets disabled.

If you enable the add-in from within Outlook > Options > Add-ins > COM Add-ins, it will revert back to "unloaded" when you next log on...maybe this Microsoft's way of forcing everyone to Teams, I dunno, but they need to fix it!

Here's what I'm talking about: in Outlook, only the Teams add-in will be shown when setting a new meeting: 

Outlook Teams Only

To fix this, you need to hack the registry to force Outlook to show both Teams and SFB.

To manually set it, navigate to:


The LoadBehavior key will most likely be set to "0" which is Off.

LyncAddin LoadBehavior Off

We need to set the LoadBehavior key value to "3" which is Load At Start:

LyncAddin LoadBehavior On
If you don't have the key present (sometimes it won't be there), or don't want the hassle of setting it, you can grab the reg file from my Google Drive. Installing this will create the key if it's missing, or set it to "Load At Start" if it's in there already.
You can also use PowerShell to set the key, by running the following cmdlet in an Elevated PS session:
Set-ItemProperty -Path "HKCU:\Software\Microsoft\Office\Outlook\Addins\UCAddin.LyncAddin.1" -Name loadbehavior -Value 3
There's no need to restart the machine with either the reg change or the Shell method, the changes will automatically take effect.
Now, when creating a new meeting, both add-ins will be shown:
Outlook SFB And Teams

You can push out the reg file in a GPO if you have a lot of machines that are affected - in my case it was only a few of us.

Saturday, March 23, 2019

Office 365 Create Remote PowerShell Shortcut

In a previous post, I showed how to create a Remote PowerShell shortcut for Exchange on-premises, to save you from having to type in the remote session every time you connect.

Since I've been doing more work in O365, I decided to do the same for that; especially because Office 365 has many more connections you have to run, such as Exchange, Skype For Business/Teams, Azure AD, and Security Center.

This PS shortcut will install and import those sessions and get you signed in all in one shot.

First, set your PowerShell execution policy - I use Unrestricted but you can use RemoteSigned:

Open PowerShell as admin, and run:

Set-ExecutionPolicy Unrestricted

Next, enable PS remoting by running:


Then, install the required modules:

For MSOL, run:

Install-Module MSOnline

**Note** We're using MSOL because it's more comprehensive than AzureAD

**Note** Running the above cmdlet should install the latest version straight from the PowerShell gallery. If it doesn't, browse here and grab it:


Next, download the Skype for Business Online Connector module from here:


**Note** As of this writing, the SFB Online Connector will manage Teams as well.

Next, copy the following block and paste it into Notepad:

$credential = Get-Credential
$exchangeSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri
https://outlook.office365.com/powershell-liveid/ -Credential $credential -Authentication "Basic" -AllowRedirection
Import-PSSession $exchangeSession -DisableNameChecking
$SccSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri
https://ps.compliance.protection.outlook.com/powershell-liveid/ -Credential $credential -Authentication "Basic" -AllowRedirection
Import-PSSession $SccSession -Prefix cc
Connect-MsolService -Credential $credential
Import-Module SkypeOnlineConnector
$sfboSession = New-CsOnlineSession -Credential $credential
Import-PSSession $sfboSession

**Note** I left out the SharePoint Online connection, because in order to run it without errors you either need to set your local DNS to Google's, or use the web login and that's a pain - plus I don't manage SharePoint, so......

**Note** The Exchange Online and Security Center won't work if you run MFA. For that, you need to install the EXOPS modules, which can't be run in a single window.

Save it as a .ps1 with a name like O365-Remote.ps1 to somewhere like C:\Scripts

Next, create a PowerShell shortcut anywhere, like on your Desktop:

Right-click the Desktop > New > Shortcut

New Shortcut

In the location field, enter:


PowerShell Shortcut

Click Next

Give it a name like O365RemotePS and click Finish.

O365 Remote Shortcut

Right-click the new O365RemotePS shortcut, and go to Properties.

In the Target field, add the following to the end of the line:

-NoExit -File "C:\Scripts\O365-Remote.ps1"

It will look like so:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoExit -File "C:\Scripts\365-Remote.ps1"

Click OK.

Run your new shortcut as admin, and you'll get a creds prompt for your Office 365 organization.

**Note** Enter creds in the username@orgname.com format.

Once it starts the remote session, you'll be able to run your O365 cmdlets.

Remote MSOL

Remote CSUser
Remote Mailbox