SFB Invoke Failback Error: Cannot Open Service

We recently updated our Skype For Business environment to CU7 (July 2018 Update). Since we have such a large SFB organization (20+ servers) I and another admin tag-teamed the install so it wouldn't be an all day affair.
When it came time to run the Invoke-CSComputerFailback cmdlet, the other admin was hit with the following error in the Skype For Business Management Shell:

WARNING: Invoke-CSComputerFailback failed.
WARNING: Detailed results can be found at
"C:\Users\admin2\Appdate\Local\Temp\2\Invoke-CSComputerFailback-f41aadf1-3d3f-49f1-a0bf-b4edaa5981a2.html"
Invoke-CSComputerFailback : Command execution failed: Cannot open RTCXMPPTGW service on computer "SFB-FE1.exchitup.com".
At line:1 char:1
+ Invoke-CSComputerFailback -Computer SFB-FE1.exchitup.com
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 + CatagoryInfo  : InvalidOperation: <:> [Invoke-CSComputerFailback], InvalidOperationException
 + FullyQualifiedErrorId : ProcessingFailed,Microsft.Rtc.Management.HADR.ComputerFailover.InvokeComputerFailbackCmdlet

If you browse to the location of the logfile in the error, the XML and HTML reports show the following, respectively:

XML:

<?xml version="1.0" encoding="UTF-8"?>
-<CsMgmtLog Name="Invoke-CsComputerFailBack">
<Info Time="2018-10-27 17:31:33Z" Title="Connection">Data Source=SFB-FE2.exchitup.com;Failover Partner=SFB-FE3.exchitup.com;Initial Catalog=xds;Integrated Security=True;Application Name=Microsoft.Rtc.Management</Info>
<Info Time="2018-10-27 17:31:37Z" Title="Host">SFB-FE1.exchitup.com</Info>
<Info Time="2018-10-27 17:31:37Z">Enabling services at SFB-FE1.exchitup.com...</Info>
<Error Time="2018-10-27 17:31:37Z" Title="Error">An error occurred: "System.InvalidOperationException" "Cannot open RTCXMPPTGW service on computer 'SFB-FE1.exchitup.com'."</Error>

-<Exception Time="2018-10-27 17:31:37Z" Message="Cannot open RTCXMPPTGW service on computer 'SFB-FE1.exchitup.com'." Type="InvalidOperationException">
<StackTrace Time="2018-10-27 17:31:37Z"> at System.ServiceProcess.ServiceController.GetServiceHandle(Int32 desiredAccess) at System.ServiceProcess.ServiceController.get_ServiceHandle() at Microsoft.Rtc.Management.Deployment.Core.NTService.set_Mode(StartMode value) at Microsoft.Rtc.Management.HADR.ComputerFailover.InvokeComputerFailBackCmdlet.EnableServices(DeploymentContext context) at Microsoft.Rtc.Management.HADR.ComputerFailover.InvokeComputerFailBackCmdlet.InternalProcessRecord() at Microsoft.Rtc.Management.Deployment.DeploymentCmdlet.CmdletProcessRecord()</StackTrace>

-<Exception Time="2018-10-27 17:31:37Z" Message="Access is denied" Type="Win32Exception">
<StackTrace Time="2018-10-27 17:31:37Z"/>
</Exception>
</Exception>

HTML:

The Fix:

Run the Skype For Business Management Shell as administrator, then  try the cmdlet again.

**Note** I always set the shortcut for the Shell to Run as Admin, so I don't have to do that every time. I wish it just would out of the box since I am, you know, an admin :)

Right-click shortcut, Properties > Advanced button > Run As Administrator > Ok

Now your cmdlets should run successfully and you can finish your updates!

Exchange 2016 - Publishing REST/OData For API With Kemp Load Balancer

Our company is designing an app for users to have an all-in-one-do-everything portal that will incorporate a mail client that connects to our on-prem Exchange. The app utilizes the Outlook Mail REST API to grab messages from the user's mailbox.

**Note** Office365 supports REST by default; it wasn't until Exchange 2016 CU3 that it was available for on-prem (go figure).

You can find the details here:

https://blogs.technet.microsoft.com/exchange/2016/09/26/on-premises-architectural-requirements-for-the-rest-api/

If you're running on-prem, you should be patched up way beyond CU3, right?

The issue is: REST API is enabled on Exchange 2016 CU3+ but if you run a load balancer, like Kemp's super-awesome Load Master, the API won't be published out of the box.

By default, the Kemp Exchange templates don't enable Exchange API to be published. As a workaround, you can enable the "default" content rule on your SubVSs, but some IT security departments won't like that, because it's a blanket rule.

What we'll do instead, is create a Custom Content Rule, and apply it to our OWA SubVSs ('Sub-Virtual Services), which will allow the API to be published.

Create The Rule:

In your Load Master WUI, go to Rules & Checking > Content Rules > Create New Rule:

Give it a name like "REST_API"

Rule type: leave as default (Content Matching)

Match type: leave as default (Regular Expression)

Match string: type in the following:

 /^\/api.*/

Tick Ignore Case

Click Create Rule:

Your new rule will be listed:

Adding The New Rule:

Next, we'll add our new content rule to our OWA SubVSs

Navigate to Virtual Services > View/Modify Services

Click Modify next to your Internal Service:

**Note** You can modify your external service if you want; it would just depend on your security requirements. For instance, our IT Security Department doesn't understand Exchange and therefore won't allow me to publish OWA externally, so I can only publish the API internally as well.

Expand SubVSs, and click the Rules button next to OWA:

In the drop-down, select our new "API_REST" rule and click Add:

 

 

You'll now see the API_REST rule listed for the OWA rule set:

 

 

 

Testing The New Rule:

 

In a web browser navigate to your Exchange namespace with the API URL at the end, like so:

 

https://mail.exchangeitup.com/api/v2.0/me/messages

 

You will be presented with a Save/Open JSON prompt:

 

 

Open in Notepad, and you'll see your messages in your mailbox:

 

 

Now your on-premises Exchange API REST/OData is published, and the app (or whatever needs that service) can hit it!