How to generate a sample data test file using PowerShell

5gb sharepoint drag and drop

Using PowerShell, you can quickly create a test file on your computer. I use these for testing SharePoint attachment upload sizes.

You can generate files in mb, gb, etc. easily. This example generates a 5gb test file in  C:\Users\MyUserName\AppData\Local\Temp called testfile.txt

$path = "$env:temp\testfile.txt"
$file = [io.file]::Create($path)
$file.SetLength(5gb)
$file.Close()
Get-Item $path

5gb test file

Example of the above file uploaded to SP using drag and drop:

5gb sharepoint drag and drop

SharePoint Online- Bad service principal breaks SharePoint Admin API Access “Global Service Principal ID Error: Error: {}” Resolved

Could not retrieve Global Service Principal ID Error

We recently ran into an issue when creating SPFX Azure Active Directory App registrations using an O-Auth Implicit flow that our SharePoint Admin Center API Access page then gave an error:

Could not retrieve Global Service Principal ID Error: Error: {}
Could not retrieve Global Service Principal ID Error: Error: {}
Could not retrieve Global Service Principal ID Error: Error: {}

And when using O365 CLI, I was getting the same error trying to add a new serviceprincipal or list them:

o365$ spo serviceprincipal grant list
Error: Another object with the same value for property identifierUris already exists.

Error: Another object with the same value for property identifierUris already exists.
Error: Another object with the same value for property identifierUris already exists.

So now I am completely locked out of interacting with SharePoint Online to Azure AD Applications until this is fixed.

I opened a case with SharePoint Online MS Support and they said its on the Azure side but they would look into it for me.

After a few days they got back to me and said it was due to a bad Azure AD App Registration tagged under SharePoint that is causing the error. GREAT! I now have a place to start.

  1. Go into Azure Active Directory and go to App Registrations and search for “SharePoint”. Your “broken app” is somewhere in one of these two registrations.

    App registrations for SharePoint
    App registrations for SharePoint
  2. Big thanks to this article where it says how to find the bad one, https://github.com/SharePoint/sp-dev-docs/issues/3891#issuecomment-494868401 
    In the API Access admin page of SP admin https://yourcompany-admin.sharepoint.com/_layouts/15/online/AdminHome.aspx#/webApiPermissionManagement, go to view source and search for spfx3rdPartyServicePrincipalId. You will find a GUID, THIS IS THE GOOD ONE. The other application is bad.
    Good one (found in view source of Admin page- “spfx3rdPartyServicePrincipalId”:”ebxxxx-xxxxx-xxxxxxx899d1
  3. No go back into app registrations and map out what GUIDs belong where
    Good ID: “SharePoint Online Client Extensibility Web Application Principal
  4. That means the other one is bad! “SharePoint Online Client Extensibility Web Application Principal Helper
    According to the above article, deleting this one fixes it. AND IT DID!
    Object ID 218 is the good one, and the top one 421 is BAD!
  5. Sure enough, after removing the bad application I can now access the API Access page!

So after removing every single bad VM, app registration or enterprise app, it all comes down to removing the bad “SharePoint Online Client Extensibility Web Application Principal Helper” app registration above. Now everything works as expected and I can approve my app requests!

Big lesson learned!

Now I can do a Enable-SPOTenantServicePrincipal without errors:

Hope this helps and leave any comments below on your experience with this error.

SharePoint Online- Create a new Home Site in an existing tenant. Spoiler- its not available yet.

This week at #MSIgnite, there has been a lot of talk about Home Sites. This was released a few months back as a PowerShell command to transform any modern communication site collection into the O365 SharePoint Online landing page for your organization.

More can be read here: https://techcommunity.microsoft.com/t5/Microsoft-SharePoint-Blog/SharePoint-home-sites-a-landing-for-your-organization-on-the/ba-p/621933

SharePoint home sites are the landing sites for your organization that bring together news, events, content, conversations and video to deliver an engaging experience that reflects your voice, your priorities, and your brand.  Home sites are built on top of familiar communication sites.

UPDATE- SPO Home Sites is not available to the public yet, even on first release tenants. I will update this post once its released. See error command at last step in this post.

1- Create the site

Create a Modern Communication Site Collection that will be your hubsite, if you do not have one already.

1- create new communication site

Give the new site a name, I choose topic and Home Site:

2- new communication site title and url

2- Share the site with users

I shared with everyone except external users

3- share new communication site

3- Set the new site collection as the home site

I am hoping this does a bunch of magic to make it look like the Look Book’s rollup.

Enter the URL of your newly created site collection in the URL below:

Set-SPOHomeSite -HomeSiteUrl https://yourtenant.sharepoint.com/sites/HomeSite

andddd error!

Set-SPOHomeSite : The requested operation is part of an experimental feature that is not supported in the current
environment.

4- set-spohomesite error experimental feature not supported

Wow, that was unexpected.

I have not seen a release date yet, but wanted to post this in hopes it will save someone else the time of creating a site.

Update as of 7/3/2020-

Using 16.0.20212.12000 of SPO Management shell- slightly different error:

 

PS C:\WINDOWS\System32> Connect-SPOService

cmdlet Connect-SPOService at command pipeline position 1
Supply values for the following parameters:
Url: https://m365xIDHERE-admin.sharepoint.com/
PS C:\WINDOWS\System32> Set-SPOHubSite

cmdlet Set-SPOHubSite at command pipeline position 1
Supply values for the following parameters:
Identity: https://m365xIDHERE.sharepoint.com/sites/Eric-PublicTeam
Set-SPOHubSite : Cannot invoke method or retrieve property from null object. Object returned by the following call
stack is null. “GetHubSitePropertiesByUrl

At line:1 char:1
+ Set-SPOHubSite
+ ~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Set-SPOHubSite], ServerException
+ FullyQualifiedErrorId : Microsoft.SharePoint.Client.ServerException,Microsoft.Online.SharePoint.PowerShell.SetSP
OHubSite

PS C:\WINDOWS\System32> Get-SPOHubSite
PS C:\WINDOWS\System32>

Cannot invoke method or retrieve property from null object. Object returned by the following call stack is null. "GetHubSitePropertiesByUrl"

Azure VM Remote Desktop error- CredSSP encryption oracle remediation

I ran into an issue where I could not Remote Desktop to my Azure VM and was getting this error:

Remote Desktop Connection
An authentication error has occurred.
The function requested is not supported

Remote computer: 13.64.xx.xxx
This could be due to CredSSP encryption oracle remediation.
For more information, see https://go.microsoft.com/fwlink/?linkid=866660

The issue was a March 2018 security update patch that was applied to my CLIENT computer, so I cannot connect to the Azure SERVER Windows Server 2012 R2 Datacenter VM that has not had updates applied recently.

https://blogs.technet.microsoft.com/mckittrick/unable-to-rdp-to-virtual-machine-credssp-encryption-oracle-remediation/

Examples:
1.       If the client is updated and you try to RDP to an Azure VM that was not updated, then it will be blocked and see the error message.
2.       If the client is not patched while server is updated, RDP can still work. But the session will be exposed to the attack.
3.       If both client & server are patched with default setting (Mitigated), RDP will work in a secure way.

I really could not find a solution anywhere that worked. Just a bunch of links to archaic Windows Updates websites from the early 2000’s (reminds me how far we came)

Solution

The solution is to Uninstall the KB on your CLIENT computer so you can install the KB on your SERVER VMs, then you can reinstall the KB on your CLIENT if you would like.

 

Detailed Steps

  1. I UNINSTALLED the KB4103721 from my Windows 10 machine, so I could patch my VM’s and rebooted.
  2. I RDP’ed from my CLIENT to each SERVER that was not patched, and installed the patch (note a different OS so a different KB patch)
  3. Applied Installed a KB on each SERVER and rebooted http://www.catalog.update.microsoft.com/Search.aspx?q=KB4103715
  4. On my CLIENT Windows 10 PC, I reinstalled the KB
    1. Go to https://portal.msrc.microsoft.com/en-us/security-guidance/advisory/CVE-2018-0886 and find your OS, then click Security Update
    2. Continue to download the appropriate patch (https://www.catalog.update.microsoft.com/Search.aspx?q=KB4103721)
    3. Once downloaded, install the KB

That’s it! I could not find an easy explanation of what KB to uninstall and which one to install.

A side note, if I enabled automatic updates, I think this wouldn’t have happened. My SharePoint VMs are not patched automatically since they are Dev machines.

Hope this helps.

Deleting a large list from SharePoint using PowerShell

I recently ran into an issue trying to delete a list over 5,000 items from SharePoint. I tried using Metalogix Content Matrix to delete the list/site, but they all were bound by the threshold. I realized I had to use PowerShell, but research lead to even this having issues deleting the list. The solution was batches of 1,000 items. It takes a few hours to remove 25,000 items, but I was able to delete the list once the items were removed.

I received this error prior to deleting the items in the list using a script via modern UI Site Contents menu:

“My List- 24453 items- We’re sorry, we had some trouble removing this. You can try again from the settings page.”

Similar error on the list settings page:

“The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator.”

This script on MSDN was able to remove the items so I could then delete the list in the browser: https://blogs.msdn.microsoft.com/ahmedamin/2017/08/03/bulk-delete-sharepoint-items-in-a-large-list-using-powershell/

Here is the script as well:

[code language=”powershell”]

Add-PSSnapin Microsoft.SharePoint.Powershell -ea SilentlyContinue
$web = get-spweb “https://intranet/sites/home”
$list = $web.lists[“My List Title Here”]
$query = New-Object Microsoft.SharePoint.SPQuery
$query.ViewAttributes = “Scope=’Recursive'”
$query.RowLimit = 1000
$query.ViewFields = “”
$query.ViewFieldsOnly = $true
do
{
$listItems = $list.GetItems($query)
$query.ListItemCollectionPosition = $listItems.ListItemCollectionPosition
foreach($item in $listItems)
{
Write-Host “Deleting Item – $($item.Id)”
$list.GetItemById($item.Id).delete()
}
}
while ($query.ListItemCollectionPosition -ne $null)

[/code]

This above script was able to remove the items, then I could delete the list.

All done!

GoDaddy’s domain expiration process explained

I had a good call with GoDaddy today to explain the process of what happens to a domain when it is deleted. I was interested in purchasing a domain that I know was deleted today. Of course, the easiest way is to have the account holder un-delete the domain and transfer it, but in this case the user who deleted it no longer wants to deal with the domain, or me asking to have it transferred (it’s a two-step process and takes time).

When a domain is deleted, the account holder has a few weeks to undo that action. After this time, the domain goes to an expired domains auction. If no one buys it, GoDaddy backorder customers ($25-$35 cost for a backorder, good for a 1-year registration) will purchase it. If you do not have a backorder, you cannot purchase the domain during this time since you are just a normal person. The domain then goes to a secondary auction. The secondary auction expires about 84 days from when the domain was first deleted by the user. That’s why the GoDaddy support team says “about 90 days the domain could be released back for normal purchase” if no one buys it at auction. GoDaddy monitoring on a domain just sends you an email of the status. I suppose you could order a backorder during the auction process if you see no one is purchasing it, then when it closes and goes to backorders you might be able to get it that way.

Here is the time-table for the scenario of what happens to a domain when a user deletes it:

Day Status

0

User deletes domain

26

Expired Domain Auction starts

36

If someone bids, done, domain is sold.
If no one bids, it goes to the first person who Backordered the domain. You cannot simply buy the domain for $9.99, you need to purchase a backorder.

41

3-day floating period the domain is in limbo. Nothing happens here.

43

Secondary Domain Auction starts

84

The domain is released for normal purchase if it didn’t sell

 

So, if the domain you are looking for is any good, it might sell at auction. Otherwise, purchase a backorder for $25-$35 and hope you get it! If it sells in the auction, you are out the $ for the backorder, so it’s a gamble. In this case, I don’t want to pay (gamble) that no one bids and I will get the domain, since I am just interested in trying to list it for sale myself, so I am passing on purchasing a backorder and just letting it go. I might check back in 84 days but I have a felling it will be purchased.

A good rule of thumb I have noticed is if the domain is worth anything, some domain selling company will snatch it up and you will never see it again. Still waiting for ericschrader.com, I have been waiting for this to fall out of auctions and auction resellers for about 5 years now.

But hey, you might get lucky. I let SharePointEric.com expire a few years ago and it was bought at auction by a reseller, and now it’s back for normal purchase for $11.99 from GoDaddy:

GoDaddy has a cool apprasal tool to check the monetary value of a domain: https://www.godaddy.com/domain-value-appraisal/appraisal/?checkAvail=1&tmskey=&domainToCheck=sharepointeric.com

If you have any tips, post them below.

There is so much to purchasing domains now days, its not like the old days (back before emoji’s).

SharePoint Online Communication site – How to share with external users

I had an issue where when I tried to share my SharePoint Online Communication Site with an external user, I received an error:

Your organization’s policies don’t allow you to share with these users. Go to External Sharing in the Office 365 admin center to enable it.

But when I went to my O365 tenant, sharing was enabled:

Allow users to invite and share with authenticated external users
Allow sharing to authenticated external users and using anonymous access links

The issue is that sharing is disabled on the site, so we need to use PowerShell to fix this.

Solution

As a SPO tenant admin, open PowerShell and do Connect-SPOService, then enter your tenant-admin.sharepoint.com URL. Supply your global admin credentials.

Then run get-sposite with your site.sharepoint.com URL, but pipe (|) the results to format-list (fl) with the property attribute looking for shar (for sharing)

 

Connect-SPOService

Url: https://tenant-admin.sharepoint.com

Get-SPOSite https://tenant.sharepoint.com/sites/Extranet | fl -prop *shar*

DisableSharingForNonOwnersStatus :

SharingCapability : Disabled

SiteDefinedSharingCapability : Disabled

DisableCompanyWideSharingLinks : NotDisabled

SharingDomainRestrictionMode : None

SharingAllowedDomainList :

SharingBlockedDomainList :

 

So set the sharing to 1 (ExtenlaUserSharingOnly)

set-SPOSite https://tenant.sharepoint.com/sites/Extranet –SharingCapability 1

Refresh the page you want to share and now the external user invite is allowed.

Done! Now you are able to share a modern SharePoint Online site with guest users. A better approach I use is to create a group and share with external users, that way security and content are isolated from employee content.

SharePoint Online blog site- How to edit the homepage

Problem

After about a week of troubleshooting the classic SharePoint Online blog subsite template (BLOG#0), I was FINALLY able to figure out why I could not edit the homepage.

Solution:

There is a site feature called “Site Pages” that needs to be activated. Once activated, you can edit the blog homepage (assuming you have permissions).

Figure 1- Activate the Site Pages feature on your Blog Site in SharePoint Online. Now you can edit the blog homepage!

Figure 2- SharePoint Online Blog Site Edit button fixed

BAM!

Figure 3- The SharePoint Online Classic Blog template is now editable for the homepage. The About this blog image can be removed.

I was lucky to run across this when troubleshooting why a modern SPFx extension would not show on a classic homepage.

Hope this helps!

Similar issues:

SharePoint Online- Enable New Modern experience on root site collection

UPDATE 9/27/2019- Microsoft this month is releasing a way to “Swap” sites. So create a new modern site collection, then swap it with the root. Invoke-SPOSiteSwap https://docs.microsoft.com/en-us/powershell/module/sharepoint-online/invoke-spositeswap?view=sharepoint-ps#description

If the target is the root site at https://tenant-name.sharepoint.com, then the following preparation activities should be performed prior to performing the swap:

  1. Any Featured links defined in SharePoint Start Page at https://tenant-name.sharepoint.com/_layouts/15/sharepoint.aspx will not be displayed after performing the swap. If required, the Featured links should be documented so they can be manually recreated after the swap.
  2. Functionality such as external sharing and application interfaces are dependent on the policies and permissions defined at the root site. Review the source site to ensure that it has the required policies and permissions as per the existing root site. This includes external sharing settings as well as site permissions.

UPDATE 1/30/2019- Still waiting for the below MS Ignite command. While we wait we can try Jeff Jone’s approach of forcing creation of a modern communication site using the classic site wizard with a client side developer trick: https://www.spjeff.com/2018/12/31/video-create-modern-communication-for-root-site-in-tenant/

UPDATE 9/27/2018- At #MsIgnite, Microsoft just announced a way to convert the root site into a modern communication site using PowerShell!
https://twitter.com/jeffteper/status/1045159986291200000?s=20

Enable-CommSite -url https://yourtenant.sharepoint.com $username [email protected] $password puppies123

Note: this might be the tenant admin url? https://yourtenant-admin.sharepoint.com

This new PS command is not yet publicly available. We just demo’ed it at this session in Ignite. (link: https://myignite.techcommunity.microsoft.com/sessions/65744) myignite.techcommunity.microsoft.com/sessions/65744. We hope to start rolling this out to customers by the end of 2018. Thank you for the enthusiasm and interest.

My old post-

I really enjoy the new Modern experience of SharePoint Online communication sites; however, this requires creating a new SharePoint Online site collection at /sites/new site for the path. The client requested to have the root site branded with the Modern experience. However, while I was able to get the page to appear as the modern experience, I could not match it to the new modern communication sites template 100%. Please post any comments if you have any suggestions to convert the root site to match the communication site look and feel.

SharePoint Online Admin center settings

In the SharePoint Online Admin Center, make sure these settings are all default:

SharePoint Lists and Libraries experience New experience (auto detect)

SharePoint Online root site settings

Next, navigate to your root site, yourcompany.sharepoint.com.

Note how the Modern Experiences is only enabled on lists and libraries by default now days:

To create your first modern page, go to your Pages library. From here, you can create a new “Site Page” which contains all of the new modern page experiences:

Now you can add modern web parts, edit layouts, etc.:

Once finished, publish the page.

Setting modern page as homepage

Go to the pages library, then set the new modern page you published as the homepage by clicking “Make homepage”:

Removing left Quick Launch navigation, attempting to match a classic site to a modern communication site (fail)

The page is now modern, but the quick launch is showing. Modern communication sites do not have this.

This is not easy, unless you want to cheat with CSS. But my goal is to replicate the OOTB new Modern communication site experience on classic pages.

I compared the Site Features between a Classic site and a New Modern Communication Site:

The classic site has the following Site Features activated:

Classic Feature Status
Getting Started Active
Mobile Browser View Active
SharePoint Server Enterprise Site features Active
SharePoint Server Publishing Active
SharePoint Server Standard Site features Active
Site Feed Active
Site Notebook Active
Team Collaboration Lists Active
Wiki Page Home Page Active

Maybe some of these features are the culprit, but nothing stood out.

When I compared the navigation, I found that the Communication sites use Current navigation across the top.

I thought this can be set by swapping the masterpage from “Seattle” to “Oslo” on the Classic site, but it did not affect the modern page I created on the classic site. Crazy.

I also noticed there is no Full Width web part on the modern experience:

So as close as you can get it OOTB is:

Update- how to add a full width web part

If you want to add full width content, you can insert a full-width layout/section to your modern page:

Once you add the full width section, you can add a Hero or Image web part. I dont like these, so I created a custom SPFx jQuery Bootstrap carousel and found a hack to allow the custom web part to appear in this special full width region:

SPFx web part full width hack:
https://blog.velingeorgiev.com/how-add-spfx-webpart-full-width-column

Then, I can add custom HTML/CSS/jQuery to the full width region.

I sometimes copy the home.aspx over and over for subpages so I don’t get the big ugly image banner like I would get OOTB on subpages.

Here is an example of something our team has worked very hard on. I have 1 SPFx web part for the carousel (full width hack) and a SPFx extension for the footer. I sanitized it a lot and excluded a lot of the other web parts due to the data:

SPFx SharePoint full width web part carousel slider

Closing thoughts

If anyone knows how to move the left Quick Launch to under the Site title to match the Communication site template, let me know. Also post any comments about other differences you find between the Modern page on a classic site vs the new Modern communication sites (aside from the O365 group).

SharePoint Server Trail period reinstall fix

Today, I was hit with a SharePoint error on a developers VM when I tried to create a new Site Collection in Central Admin:

Sorry, something went wrong

The trial period for this product has expired.

When I tried to view a teamsite homepage, I received this error:

Sorry, something went wrong

The “SiteFeedWebPart” Web Part appears to be causing a problem. Object reference not set to an instance of an object.

Web Parts Maintenance Page: If you have permission, you can use this page to temporarily close Web Parts or remove personal settings. For more information, contact your site administrator.

Obviously, things were not looking good for me.

I tried rebooting, IIS resets, PSConfig, dismounting my content DB and creating a new one, all with no luck.

I thought perhaps it was because I reset my service account passwords (back to the same password due to a time crunch recently) and maybe that was causing the service accounts to have issues, but I didn’t think that was the problem.

Credit to other bloggers

Luckily, I found a few blogs that reminded me I used a SharePoint Trial media to install these developer VMs. AAH HA! But, I cringed thinking I would have to dismount my farm db, reinstall my binaries, and face any challenges from that potential nightmare process. I WAS ABLE TO SOLVE IT!

First, this blog post showed me where to upgrade the SharePoint product key. I went on MSDN, got my key, and updated my version. They key I had previously was giving an error, so try a few different versions of your keys. https://blog.devoworx.net/2017/02/25/extend-sharepoint-trial-period/

Second, this blog post had me run PSCONFIG with a better flag to reset security, http://alstechtips.blogspot.com/2014/01/error-trial-period-for-this-product-has.html

How to fix these errors

First, go to Central Admin > Upgrade and Migration > Convert Farm license type.

Enter your SharePoint Server 2013 product key from MSDN. In my case, I used SharePoint Server 2013 with Service Pack 1- Enterprise key from MSDN.

Note: If you try downgrading Enterprise to Standard, you get this error, so use an Enterprise Key:

This product key cannot be used to convert SharePoint Server Trial with Enterprise Client Access License to SharePoint Server with Enterprise Client Access License.

Once it takes, you will get a success message:

Part 2-

Now that the key is set, you need to run PSConfig, but with a parameter.

Launch command prompt (or PowerShell) as administrator, change your directory to where PSCONFIG.exe is located, mine is in “C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\BIN”

Run the following command:

.\psconfig -cmd secureresources

It takes about 10 minutes or so.

Finally, kick off an IISReset:

DONE! Now you can create a site collection in Central Admin:

You SharePoint farm is no longer in trial.

Please post any comments below.

SharePoint Server- Renewing SSL certificates quickly

A week ago, my wildcard SSL certificate expired on GoDaddy. It was automatically purchased, but I still had to validate my domain and download a new IIS CER certificate request file.

My old post from a few years ago has some good info on certs, the file types, etc. https://eschrader.com/2014/09/23/sharepoint-2013-iis7-nlb-ssl-certificates-and-godaddy/

This is a quick guide.

The only issue I have with this quick renewal is that I could not export the certs as a PFX, but I was able to get them installed on the server in IIS by completing a CSR

Here are the steps:

GoDaddy automatically renews SSL certificate

GoDaddy has renewed your SSL certificate, but you have to verify your domain using a TXT record they give you (@ is the host field).

Once you verify, you can download the certificate. Note, this is a CER which is a certificate request that has to be completed in IIS.

Download the certificate for IIS

Copy and extract the zip to the server

I chose to delete my old certificates from my computers Personal certificate store.

Once removed, I go into IIS and go to Server Certificated under the machine:

Once in Server Certificates in IIS, click on Complete Certificate Request:

Change the file type to *.* (All files) and find your CER file you copied over:

Enter your certificates friendly name (mine is a wildcard, so I use *.mydomain.com):

Next, go to your SharePoint IIS web apps that use this host header (could be more than one) and edit the bindings and select the new certificate. If you see multiple, this is why I deleted them in my step above. If you get an error saying change this will leave behind an old certificate of the same name, just double check the other web applications in IIS to make sure they are set correctly. Updating one should update them all, but I always check each site in IIS.

That’s it! The certificate is update.

The bad thing is I have to repeat the IIS complete CSR steps on each machine. I would rather export the first one and import PFX certificate files to my other machines, but hey, this is how I got it to work.

Leave any comments below, thanks!

SharePoint Online global navigation across site collections, with highlighting and security trimming

One common request when working with SharePoint sites is having a consistent navigation across multiple site collections. If you are using a Publishing Portal site template, you can use the Managed Navigation for your Global Navigation (or top navigation). This also supports drop downs. I did a quick test and it appears to support highlighting of the current element, which is nice considering the URLs are hard coded rather than dynamically added.

As for security, MS indicates the term store navigation supports security trimming as follows: “Note that if users don’t have access to the physical .aspx page (read permissions at least), the link won’t appear in menus even if these options are checked. By this way you can also control links displayed to users according to permissions. It follows the same behavior as the default SharePoint navigation menus.”

The drawback: You have to create a term set in each site collections Managed Navigation, pin EACH of your parent term navigation items (but it includes child terms at least). Its a lot of work, but the only way without custom code. Other options I have seen discussed are using Search web parts or CSOM, etc. Possibly 3rd party solutions. This does not work on new modern team sites (at least at the time of writing this), I get “access denied” when trying to enable Managed Navigation, even after turning on Publishing for the site/web.

Managed Navigation:

Under the target site collection(s), configure your navigation from Site Settings

Ensure Managed Navigation is checked under Global Navigation:

Uncheck:

Next, rather than creating a new term set from the site collection, do it in SharePoint Admin Center.

Go to the Admin tile:

Go to SharePoint under Admin Centers:

Select Term Store on left navigation:

Add your organizational tenant email to Term Store Administrators, save and reload the page.

Then, select the root term store for your O365 tenant, and select New Group:

Type in the orange input box, call it Navigation or something unique:

Select your new term group, and add yourself as a Group Manager and Contributor:

Create new Term Set under the group:

I just called mine Sites, but this is the actual element you will be selecting for your navigation. All child terms will appear in the actual navigation menu.

Then, select the sites element and add yourself as Owner, Contact and Stakeholder and SAVE:

Go to the Intended Use tab at the top, and enable “Use this Term Set for Site Navigation”:

Note: I also see faceted navigation, which IF the product catalog is now possible in SharePoint Online I will do another post soon, as I have been waiting years for this. I remember the roadblock was something with search managed properties…

Then under your term set, add your terms by selecting Create Term:

Go to Navigation tab and add your custom link:

You can create sub terms under terms as well to enable a drop-down navigation.

You can re-order terms in a group by selecting the group and going to Custom Sort:

Now just repeat the first step of selecting Managed Navigation and the Term Group on each of your site collections you want to inherit this navigation.

Update: Selecting this term set is limited to 1 per site collection. So the workaround acording to MS is to “Pin” each of your primary terms (with children) to the new site collections term set. https://support.microsoft.com/en-us/help/3144166/implement-global-navigation-across-multiple-site-collections-through-managed-navigation-in-sharepoint-server-2013 see steps 5-7 One note, it doesn’t seem to preserve custom sorting from the parent term set.

Uncheck:

Done!

Note: if you see any errors (such as Error loading navigation: The Managed Navigation term set is improperly attached to the site), switch the navigation to Structural on BOTH Global and Current, SAVE the changes, then change it back to Managed (and uncheck Add new pages to navigation automatically and Create friendly URLs for new pages automatically) and the error should go away.

Uncheck:

SharePoint 2010 Content Deployment Job failed. The remote upload web request failed. The remote server returned an error: (404) Not Found.

Summary

My farm content deployment jobs had been working, but all of a sudden stopped one day. The end fix was to edit a Central Admin web.config file upload size on the target WFE servers.

Issue

I was seeing the following errors in my ULS logs after 23 minutes of packing up just under 1gb of content from QA to PRD:

  • ContentDeploymentJob.ExecuteJob(): Failed ExecuteJob() with JobInfo: Name: ‘QA to Prod Job’, Id:’5db43c5d-1c3b-41cd-ac0a-495a48acb175′, JobType:’ServerToServer’, Description:”. Exception: ‘System.Net.WebException: The remote server returned an error: (404) Not Found.
  • Failed to transfer files to destination server for Content Deployment job ‘QA to Prod Job’. Exception was: ‘System.Net.WebException: The remote server returned an error: (404) Not Found.

In central admin, I was receiving this error after 23 minutes of running the job:

  • Content deployment job ‘QA to Prod Job’ failed.The remote upload Web request failed.

Resolution

Thanks to this article, I figured out the issue was the max upload size of the Central Admin web config on the target server WFE’s: https://social.msdn.microsoft.com/Forums/sharepoint/en-US/1d4aca49-40c1-414e-980e-150b148caf10/content-deployment-problems?forum=sharepointgeneralprevious

This is what finally worked for me, On the target WFE(s) modified web.config same as above:

  1. On target WFE(s), Edit the web.config file located in C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\ADMIN\Content Deployment

    <httpRuntime maxRequestLength=”307200” />

    <requestLimits maxAllowedContentLength=”314572800” />

    1. old values (for backup purposes)

      <httpRuntime maxRequestLength=”102400″ />

      <requestLimits maxAllowedContentLength=”104857600″ />

  2. On target WFE(s), perform an IISreset
  3. Launch the CDP job again and it should run.

Automating Azure IaaS SharePoint VM Provisioning via PowerShell Remoting

I have been searching for a way to rapidly create standalone Developer SharePoint 2013 standalone VM’s joined to a central domain for our in-house developers. Our team has created 60+ SharePoint VMs on Azure and continue to create about 10 per month. We are beginning to treat our VMs needing hours of repair like cattle, and no longer like the family dog if they have issues or “get terminally ill”, VM’s are replaced with a brand new shiny cow within 4 hoursJ

The process of manually creating VMs was not fun, taking over a day per VM in the beginning on average. I did not want to use Sysprep, since I would have to maintain multiple VM Images at a single point in time. For some, this might be the best way to go.

My solution was to create a lightweight PowerShell set of scripts that create the VM in Azure, install the applications, and keep everything consistent. I can create 4+ VMs at one time, all under ~4 hours total. This is a process that scales somewhat, to meet our needs. Perfect.

Alternatives to this manual PowerShell process I went through:

  • Sysprep would save a ton of steps, but is not as easy to update OS patches, etc. as newer software comes out
  • AzureRM– Azure Resource Manager is a lot easier. However, these topologies seem somewhat isolated and all of our existing VMs and didn’t work well with what we had in place for our network/VPN, etc.
  • Azure SharePoint QuickStart templates/images- These were preconfigured and had various OS settings changed. Similar to the above Azure RM solution issues we ran into.

Assumptions:

  • You have an Azure subscription all set up, with a virtual network/DNS/Subnet (we have a site-to-site VPN)
  • You have domain controller with all of the SharePoint service accounts created for Least Privileges security installation
  • You have installed the latest version of Azure PowerShell installed, rebooted after installing it, and performed the Add-AzureAccount command
  • You have used AutoSPInstaller before

What you need:

  • The following information from Azure
    • Subscription ID
    • Virtual Network info
      • Network Name
      • Subnet Name
    • Resource Group
  • The following pre-existing VMs:
    • DC
      • Service Accounts
    • Fileshare VM
      • All of the necessary ISO’s and EXE’s
        • Installer files:
          • 7zipInstall.msi
          • ccleaner.exe
          • fiddler4setup.exe
          • Copy-Item ‘C:\Fileshare\applications\Firefox Setup Stub 36.0.4.exe’
          • iview438_setup.exe
          • LINQPad4Setup.exe
          • npp.6.7.5.Installer.exe
          • paint.net.4.0.5.install.exe
          • PowerGUI.3.8.0.129.msi
          • cutepdf-writer.exe
          • CKS.Dev11.vsix
          • codecompare.exe
      • Stand Alone EXE’s
        • ULSViewer.zip
        • U2U.SharePoint.CQB2010.zip
      • Applications (Extracted into their own folder with configuration.ini files)
        • CamlDesigner2013
        • Visual Studio 2012
        • Visual Studio 2015
        • SharePoint Designer 2013
        • en_sql_server_2014_enterprise_edition_with_service_pack_1_x64_dvd_6669618
        • AutoSPInstaller for dev
        • sql2014config file for dev
        • en_sharepoint_server_2013_with_sp1_x64_dvd_3823428
        • SharePoint 2013 June 2015 CU (note, if you download this from the internet, uncheck the security property so you don’t get prompted during the AutoSPInstaller process for UAC- right click all 3 CU files and go to Security and unblock, you only have to do this one time on the fileshare.)
  • Silent install for software (one-time prep, then save the folder on the Fileshare VM)

Configure PowerShell variables for the new standalone developer SharePoint VM

#VM Name will be ASP13D08

#IP will be 192.168.1.87

#Cloud service Company-Redondo-D08 (each developer has their own cloud service so they can power on VMs without having to wait for the other developers to start at the same time)

#service accounts

#Single SharePoint 2013 developer VM

[code language=”powershell”]$varVMLocation = "A"
$varVMServerType = "SP"
$varVMSPVersion = "13"
$varVMType = "D"
$varVMIntanceNum = "08"
$varVMReduxSuffix = "" # I sometimes append a version letter to the end of the developers VM, if they are getting an additional VM of the same role.
$spsetupname = "svc_spsetup"
$spsetuppasstext = "passw0rdspsetup"
$users = @("svc_spsetup", "svc_spfarm", "eric.schrader", "dev1", "etc");
$varVMStaticIP = "192.168.1.87"
$varStorageAccount = "Company" + $varVMLocation + $varVMType + $varVMIntanceNum #unique
$varStorageAccount = $varStorageAccount.ToLower()
$service = "Company-Redondo-" + $varVMType + $varVMIntanceNum
$instancesize = "Basic_A4"
$subscriptionid = "12345678-12345-123456"
$subscriptionName = "Microsoft Azure Enterprise"
$imageFamily = "Windows Server 2012 R2 Datacenter" #Azure VM Image name, the latest will be used below.
$localadminname = "company.admin" #cant be "administrator", etc.
$localadminPassword = "passw0rdlocaladmin"
$joindomain = "Domain.local"
$domainname = "Domain"
$machineOU = "OU=Azure,OU=Development,OU=Servers,OU=Seattle,DC=Company,DC=local"
$timezone = "Pacific Standard Time"
$domainusername = "svc_spsetup"
$domainpassword = "passw0rdspsetup"
$datadiskGB = 127
$datadiskLUN = 0
$datadiskCACHE = "None"
$vmsubnet = "Subnet-1"
$vmaffinitygroup = "VPN-Linked"[/code]

Create the Azure storage account if it doesn’t exist, the set it as the default for PowerShell

#Get-AzureStorageAccount | ft

#Change varStorageAccount to lowercase

[code language=”powershell”]$lowerStorageAccount = $varStorageAccount.ToLower()

Try{

get-azurestorageaccount -storageaccountname $varStorageAccount -ErrorAction Stop

#if this fails to get it, it will create it below. Need above error action

}

Catch {

#you got an error trying to get it, so create it.

Write-output "creating storage account $varStorageAccount"

New-AzureStorageAccount -StorageAccountName $lowerStorageAccount -Label $lowerStorageAccount -AffinityGroup $vmaffinitygroup

}[/code]

#now that it exists, set it as default.

[code language=”powershell”]Set-AzureSubscription -CurrentStorageAccountName $varStorageAccount -SubscriptionId $subscriptionid

Select-AzureSubscription -SubscriptionId $subscriptionid -Current[/code]

Create the VM using above variables

#try to fix the DNS error in the Comapny DC, WARNING: The specified DNS name is already taken.

#New-AzureService -Label $service -Description $service -AffinityGroup $vmaffinitygroup -ServiceName $service

 

[code language=”powershell”]</span>New-AzureVMConfig -Name $name -InstanceSize $instancesize -ImageName $image | Add-AzureProvisioningConfig -AdminUserName $localadminname -EnableWinRMHttp -TimeZone $timezone -DisableAutomaticUpdates –Password $localadminPassword -WindowsDomain -JoinDomain $joindomain -Domain $domainname -DomainUserName $domainusername -DomainPassword $domainpassword -MachineObjectOU $machineOU | Add-AzureDataDisk -CreateNew -DiskSizeInGB $datadiskGB -DiskLabel $datadiskname -LUN $datadiskLUN -HostCaching $datadiskCACHE | Set-AzureSubnet –SubnetNames $vmsubnet | Set-AzureStaticVNetIP -IPAddress $varVMStaticIP| New-AzureVM –ServiceName $service -AffinityGroup $vmaffinitygroup[/code]

Configures Secure Remote PowerShell Access to Windows Azure Virtual Machines

Download PS1 file from this blog post to your local computer with Azure PowerShell. https://gallery.technet.microsoft.com/scriptcenter/Configures-Secure-Remote-b137f2fe

#CD to location in PowerShell, for example, your desktop:

[code language=”powershell”]Cd C:\users\eric.schrader\desktop[/code]

#Create WInRM Cert to new VM

[code language=”powershell”].\InstallWinRMCertAzureVM.ps1 -SubscriptionName $subscriptionname -ServiceName $service -Name $name[/code]

Connect to remote session

#Connect via remote powershell as local azure.admin
#uses variables from when the VM was created above

[code language=”powershell”]$passwordsec = convertto-securestring $localadminPassword -asplaintext -force

$user = $name +"\"+ $localadminname

$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $user,$passwordsec

$uri = Get-AzureWinRMUri -ServiceName $service -Name $name

Enter-PSSession -ConnectionUri $uri -Credential $cred

$env:computername[/code]

#Run variables Again!!!

# IMPORTANT, COPY AND PASTE THE ABOVE VARIABLES SECTION IN AGAIN. This is a new remote session to the new Azure VM.

[code language=”powershell”]$varVMLocation = "A"

$varVMServerType = "SP"

$varVMSPVersion = "13"

$varVMType = "D"

$varVMIntanceNum = "08"

$varVMReduxSuffix = "" # I sometimes append a version letter to the end of the developers VM, if they are getting an additional VM of the same role.

$spsetupname = "svc_spsetup"

$spsetuppasstext = "passw0rdspsetup"

$users = @("svc_spsetup", "svc_spfarm", "eric.schrader", "dev1", "etc");

$varVMStaticIP = "192.168.1.87"

$varStorageAccount = "Company" + $varVMLocation + $varVMType + $varVMIntanceNum #unique

$varStorageAccount = $varStorageAccount.ToLower()

$service = "Company-Redondo-" + $varVMType + $varVMIntanceNum

$instancesize = "Basic_A4"

$subscriptionid = "12345678-12345-123456"

$subscriptionName = "Microsoft Azure Enterprise"

$imageFamily = "Windows Server 2012 R2 Datacenter" #Azure VM Image name, the latest will be used below.

$localadminname = "company.admin" #cant be "administrator", etc.

$localadminPassword = "passw0rdlocaladmin"

$joindomain = "Domain.local"

$domainname = "Domain"

$machineOU = "OU=Azure,OU=Development,OU=Servers,OU=Seattle,DC=Company,DC=local"

$timezone = "Pacific Standard Time"

$domainusername = "svc_spsetup"

$domainpassword = "passw0rdspsetup"

$datadiskGB = 127

$datadiskLUN = 0

$datadiskCACHE = "None"

$vmsubnet = "Subnet-1"

$vmaffinitygroup = "VPN-Linked"[/code]

Set proper storage account in remote session

# Now that you set the variables, set the storage account for the remote session

[code language=”powershell”]Set-AzureSubscription -CurrentStorageAccountName $varStorageAccount -SubscriptionId $subscriptionid

Select-AzureSubscription -SubscriptionId $subscriptionid -Current[/code]

Format F drive for SharePoint/SQL, permission service accounts

#Format F drive

[code language=”powershell”]$labels = @("DATA1","DATA2")

Write-Host "Initializing and formatting raw disks"

$disks = Get-Disk | Where partitionstyle -eq ‘raw’ | sort number

## start at F: because sometimes E: shows up as a CD drive in Azure

$letters = 70..89 | ForEach-Object { ([char]$_) }

$count = 0

foreach($d in $disks) {

$driveLetter = $letters[$count].ToString()

$d |

Initialize-Disk -PartitionStyle MBR -PassThru |

New-Partition -UseMaximumSize -DriveLetter $driveLetter |

Format-Volume -FileSystem NTFS -NewFileSystemLabel $labels[$count] `

-Confirm:$false -Force

$count++

}

GET-WMIOBJECT –query "SELECT * from win32_logicaldisk where DriveType = ‘3’"[/code]

#add developer, and admins/spsetup/spfarm, set in $users variable above

[code language=”powershell”]foreach($user in $users) {

$domainuser= $domainname + "\"+$user

$Group = "Administrators"

$de = [ADSI]"WinNT://$name/$Group,group"

$de.Add("WinNT://$domainname/$user")

Write-Host "Done, $domainuser has been permissioned to this computer."

}

net localgroup administrators[/code]

#create folder and share for apps

[code language=”powershell”]New-Item -Path F:\tools -ItemType directory -Value Tools

New-SMBShare –Name "Tools" –Path "F:\Tools" -ChangeAccess "Everyone"[/code]

 Disable UAC (for developers), restart computer, set execution policy, etc.

#Disable UAC

[code language=”powershell”]Set-ItemProperty -Path HKLM:\Software\Microsoft\Windows\CurrentVersion\policies\system -Name EnableLUA -Value 0[/code]

#allow scripts

[code language=”powershell”]Set-executionpolicy unrestricted -force[/code]

#reboot

[code language=”powershell”]Restart-computer[/code]

#wait 5 minutes for reboot

#Reconnect to powershell

(exit, reconnect to remote powershell, re-run vars)

 Connect to remote session

#Connect via remote powershell as local azure.admin
#uses variables from when the VM was created above

[code language=”powershell”]$passwordsec = convertto-securestring $localadminPassword -asplaintext -force

$user = $name +"\"+ $localadminname

$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $user,$passwordsec

$uri = Get-AzureWinRMUri -ServiceName $service -Name $name

Enter-PSSession -ConnectionUri $uri -Credential $cred

$env:computername[/code]

#Run variables Again!!!

# IMPORTANT, COPY AND PASTE THE ABOVE VARIABLES SECTION IN AGAIN. This is a new remote session to the new Azure VM.

 

[code language=”powershell”]</span>$varVMLocation = "A"

$varVMServerType = "SP"

$varVMSPVersion = "13"

$varVMType = "D"

$varVMIntanceNum = "08"

$varVMReduxSuffix = "" # I sometimes append a version letter to the end of the developers VM, if they are getting an additional VM of the same role.

$spsetupname = "svc_spsetup"

$spsetuppasstext = "passw0rdspsetup"

$users = @("svc_spsetup", "svc_spfarm", "eric.schrader", "dev1", "etc");

$varVMStaticIP = "192.168.1.87"

$varStorageAccount = "Company" + $varVMLocation + $varVMType + $varVMIntanceNum #unique

$varStorageAccount = $varStorageAccount.ToLower()

$service = "Company-Redondo-" + $varVMType + $varVMIntanceNum

$instancesize = "Basic_A4"

$subscriptionid = "12345678-12345-123456"

$subscriptionName = "Microsoft Azure Enterprise"

$imageFamily = "Windows Server 2012 R2 Datacenter" #Azure VM Image name, the latest will be used below.

$localadminname = "company.admin" #cant be "administrator", etc.

$localadminPassword = "passw0rdlocaladmin"

$joindomain = "Domain.local"

$domainname = "Domain"

$machineOU = "OU=Azure,OU=Development,OU=Servers,OU=Seattle,DC=Company,DC=local"

$timezone = "Pacific Standard Time"

$domainusername = "svc_spsetup"

$domainpassword = "passw0rdspsetup"

$datadiskGB = 127

$datadiskLUN = 0

$datadiskCACHE = "None"

$vmsubnet = "Subnet-1"

$vmaffinitygroup = "VPN-Linked"[/code]

Install SQL pre-reqs

 #Install .net 3.5 for SQL prereq on SQL server

[code language=”powershell”]Install-WindowsFeature –name NET-Framework-Core[/code]

Now that the VM is ready for software installs, lets copy the software over. Due to the Windows “triple hop” issue of credentials, I cannot remote into the VM then copy from a 3rd remote location to the vm. I will have to RDP manually

RDP to fileshare computer as svc_SPSetup

Run variables on fileshare computers PowerShell

#Run variables Again!!!

# IMPORTANT, COPY AND PASTE THE ABOVE VARIABLES SECTION IN AGAIN. This is a new VM session.

[code language=”powershell”]$varVMLocation = "A"

$varVMServerType = "SP"

$varVMSPVersion = "13"

$varVMType = "D"

$varVMIntanceNum = "08"

$varVMReduxSuffix = "" # I sometimes append a version letter to the end of the developers VM, if they are getting an additional VM of the same role.

$spsetupname = "svc_spsetup"

$spsetuppasstext = "passw0rdspsetup"

$users = @("svc_spsetup", "svc_spfarm", "eric.schrader", "dev1", "etc");

$varVMStaticIP = "192.168.1.87"

$varStorageAccount = "Company" + $varVMLocation + $varVMType + $varVMIntanceNum #unique

$varStorageAccount = $varStorageAccount.ToLower()

$service = "Company-Redondo-" + $varVMType + $varVMIntanceNum

$instancesize = "Basic_A4"

$subscriptionid = "12345678-12345-123456"

$subscriptionName = "Microsoft Azure Enterprise"

$imageFamily = "Windows Server 2012 R2 Datacenter" #Azure VM Image name, the latest will be used below.

$localadminname = "company.admin" #cant be "administrator", etc.

$localadminPassword = "passw0rdlocaladmin"

$joindomain = "Domain.local"

$domainname = "Domain"

$machineOU = "OU=Azure,OU=Development,OU=Servers,OU=Seattle,DC=Company,DC=local"

$timezone = "Pacific Standard Time"

$domainusername = "svc_spsetup"

$domainpassword = "passw0rdspsetup"

$datadiskGB = 127

$datadiskLUN = 0

$datadiskCACHE = "None"

$vmsubnet = "Subnet-1"

$vmaffinitygroup = "VPN-Linked"[/code]

Copy the software

#Run from Fileshare as SPSetup in PowerShell

#re-run variables

#run installers
#Copy applications from local computer S drive on \\fileshare to server F drive

[code language=”powershell”]Copy-Item C:\Fileshare\applications\7zipInstall.msi -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\ccleaner.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\fiddler4setup.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item ‘C:\Fileshare\applications\Firefox Setup Stub 36.0.4.exe’ -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\iview438_setup.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\LINQPad4Setup.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\npp.6.7.5.Installer.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\paint.net.4.0.5.install.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\PowerGUI.3.8.0.129.msi -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\cutepdf-writer.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\CKS.Dev11.vsix -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\codecompare.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>[/code]

#copy exes to F:\Tools 

[code language=”powershell”]Copy-Item C:\Fileshare\applications\ULSViewer.zip -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\U2U.SharePoint.CQB2010.zip -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\CamlDesigner2013\* -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

Copy-Item "C:\Fileshare\Visual Studio 2012\*" -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

Copy-Item "C:\Fileshare\Visual Studio 2015\*" -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

Copy-Item "C:\Fileshare\SharePoint Designer 2013\*" -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse[/code]

#copy SQL to SQL server (Copy ISO CONTENTS)

[code language=”powershell”]Copy-Item C:\Fileshare\en_sql_server_2014_enterprise_edition_with_service_pack_1_x64_dvd_6669618\* -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

Copy-Item C:\Fileshare\AutoSPInstallerDev2013\* -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

Copy-Item C:\Fileshare\sql2014configdevint\* -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse -force

Copy-Item C:\Fileshare\en_sharepoint_server_2013_with_sp1_x64_dvd_3823428\* -Destination <a href="///\\$name\tools\AutoSPInstaller\SP\2013\SharePoint">\\$name\tools\AutoSPInstaller\SP\2013\SharePoint</a> -Recurse -force

Copy-Item "C:\Fileshare\SharePoint 2013 June 2015 CU" -Destination <a href="///\\$name\tools\AutoSPInstaller\SP\2013\Updates">\\$name\tools\AutoSPInstaller\SP\2013\Updates</a> -Recurse -force[/code]

Close RDP to fileshare and go back to your local computers PowerShell. We are ready to install the software

#re-run variables

#Run variables Again!!!

# IMPORTANT, COPY AND PASTE THE ABOVE VARIABLES SECTION IN AGAIN. This is a new remote session to the new Azure VM.

[code language=”powershell”]$varVMLocation = "A"

$varVMServerType = "SP"

$varVMSPVersion = "13"

$varVMType = "D"

$varVMIntanceNum = "08"

$varVMReduxSuffix = "" # I sometimes append a version letter to the end of the developers VM, if they are getting an additional VM of the same role.

$spsetupname = "svc_spsetup"

$spsetuppasstext = "passw0rdspsetup"

$users = @("svc_spsetup", "svc_spfarm", "eric.schrader", "dev1", "etc");

$varVMStaticIP = "192.168.1.87"

$varStorageAccount = "Company" + $varVMLocation + $varVMType + $varVMIntanceNum #unique

$varStorageAccount = $varStorageAccount.ToLower()

$service = "Company-Redondo-" + $varVMType + $varVMIntanceNum

$instancesize = "Basic_A4"

$subscriptionid = "12345678-12345-123456"

$subscriptionName = "Microsoft Azure Enterprise"

$imageFamily = "Windows Server 2012 R2 Datacenter" #Azure VM Image name, the latest will be used below.

$localadminname = "company.admin" #cant be "administrator", etc.

$localadminPassword = "passw0rdlocaladmin"

$joindomain = "Domain.local"

$domainname = "Domain"

$machineOU = "OU=Azure,OU=Development,OU=Servers,OU=Seattle,DC=Company,DC=local"

$timezone = "Pacific Standard Time"

$domainusername = "svc_spsetup"

$domainpassword = "passw0rdspsetup"

$datadiskGB = 127

$datadiskLUN = 0

$datadiskCACHE = "None"

$vmsubnet = "Subnet-1"

$vmaffinitygroup = "VPN-Linked"[/code]

RDP to Developer VM using svc_SPSetup and install SQL by PowerSHell.
#via SPSetup , possibly have to sign into RDP instead of remote PS. Takes 20 minutes

[code language=”powershell”]start-process F:\tools\sql\Setup.exe -ArgumentList "/q /SkipRules=VSShellInstalledRule RebootRequiredCheck /ConfigurationFile=F:\Tools\ConfigurationFile.ini /ERRORREPORTING=1 /IACCEPTSQLSERVERLICENSETERMS" -Wait[/code]

Close RDP to developer VM once SQL is done.

From local computer, Connect via remote powershell as spsetup. There is code in here to install SQL remotely, but it too has a triple hop credential issue since I use service accounts. I just RDP to the VM and install SQL via PowerShell there.
#uses variables from when the VM was created above

[code language=”powershell”]$passwordsec = convertto-securestring $spsetuppasstext -asplaintext -force

$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $spsetupname ,$passwordsec

$uri = Get-AzureWinRMUri -ServiceName $service -Name $name

Enter-PSSession -ConnectionUri $uri -Credential $cred

$env:computername[/code]

#FIX – maybe to install SQL remotely
#Invoke-Command -ComputerName $name -Authentication CredSSP -credential $cred -scriptblock {
#F:\tools\sql\Setup.exe -ArgumentList “/q /SkipRules=VSShellInstalledRule RebootRequiredCheck /ConfigurationFile=F:\Tools\ConfigurationFile.ini /ERRORREPORTING=1 /IACCEPTSQLSERVERLICENSETERMS” -Wait
#}

Set SQL max memory to 10GB, set Max Degree of parallelism to 1 (this is a huge script, maybe you can shorten it)

#Set Max degree of parallelism to 1

[code language=”powershell”]## Sets the ‘max degree of parallelism’ value to 1 for the specified SQL server instance
## Port 1433 is used if not specified
## 2012-10-08
## <a href="http://www.pointbeyond.com">www.pointbeyond.com</a>
## NOTE: This function requires at least serveradmin level permissions within SQL server
function SetMaxDegreeOfParallelism()

{

Param(

$server,

$port="1433")

$conn = new-object System.Data.SqlClient.SqlConnection

try

{

$connectionString = "Server="+$server+","+$port+";Database=master;Integrated Security=True;"

$conn.ConnectionString = $connectionString

$conn.Open()

$cmd = new-object System.Data.SqlClient.SqlCommand

$cmd.Connection = $conn

# Ensure advanced options are available

$commandText = "sp_configure ‘show advanced options’, 1;RECONFIGURE WITH OVERRIDE;"

$cmd.CommandText = $commandText

$r = $cmd.ExecuteNonQuery()

# Set the Max Degree of Parallelism value to 1

write-host "Setting ‘max degree of parallelism’ value to 1 for server $server…"

$commandText = "sp_configure ‘max degree of parallelism’, 1;RECONFIGURE WITH OVERRIDE"

$cmd.CommandText = $commandText

$r = $cmd.ExecuteNonQuery()

write-host "Success"

}

catch

{

write-host "An error occurred trying to set the MaxDegreeOfParallelism value to 1 for server $server" -Fore Red

write-host "Ensure that server and port parameters are correct and that the current user has at least serveradmin permissions within SQL" -Fore Red

}

finally

{

$conn.Close()

}

}

# Call the function passing in SQL server name/instance/alias and port number

SetMaxDegreeOfParallelism -server $name -port "1433"[/code]

#Set SQL Max memory – 3GB

[code language=”powershell”]Function Test-SqlSa {

<#

.SYNOPSIS

Ensures sysadmin account access on SQL Server. $server is an SMO server object.

.EXAMPLE

if (!(Test-SQLSA $server)) { throw "Not a sysadmin on $source. Quitting." }

.OUTPUTS

$true if syadmin

$false if not

#>

[CmdletBinding()]

param(

[Parameter(Mandatory = $true)]

[ValidateNotNullOrEmpty()]

[object]$server

)

try {

return ($server.ConnectionContext.FixedServerRoles -match "SysAdmin")

}

catch { return $false }

}

Function Get-ParamSqlCmsGroups {

<#

.SYNOPSIS

Returns System.Management.Automation.RuntimeDefinedParameterDictionary

filled with server groups from specified SQL Server Central Management server name.

.EXAMPLE

Get-ParamSqlCmsGroups sqlserver

#>

[CmdletBinding()]

param(

[Parameter(Mandatory = $true)]

[string]$Server

)

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") -eq $null) {return}

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Management.RegisteredServers") -eq $null) {return}

$cmserver = New-Object Microsoft.SqlServer.Management.Smo.Server $server

$sqlconnection = $cmserver.ConnectionContext.SqlConnectionObject

try { $cmstore = new-object Microsoft.SqlServer.Management.RegisteredServers.RegisteredServersStore($sqlconnection)}

catch { return }

if ($cmstore -eq $null) { return }

$newparams = New-Object System.Management.Automation.RuntimeDefinedParameterDictionary

$paramattributes = New-Object System.Management.Automation.ParameterAttribute

$paramattributes.ParameterSetName = "__AllParameterSets"

$paramattributes.Mandatory = $false

$argumentlist = $cmstore.DatabaseEngineServerGroup.ServerGroups.name

if ($argumentlist -ne $null) {

$validationset = New-Object System.Management.Automation.ValidateSetAttribute -ArgumentList $argumentlist

$combinedattributes = New-Object -Type System.Collections.ObjectModel.Collection[System.Attribute]

$combinedattributes.Add($paramattributes)

$combinedattributes.Add($validationset)

$SqlCmsGroups = New-Object -Type System.Management.Automation.RuntimeDefinedParameter("SqlCmsGroups", [String[]], $combinedattributes)

$newparams.Add("SqlCmsGroups", $SqlCmsGroups)

return $newparams

} else { return }

}

Function Get-SqlCmsRegServers {

<#

.SYNOPSIS

Returns array of server names from CMS Server. If -Groups is specified,

only servers within the given groups are returned.

.EXAMPLE

Get-SqlCmsRegServers -Server sqlserver -Groups "Accounting", "HR"

#>

[CmdletBinding()]

param(

[Parameter(Mandatory = $true)]

[string]$server,

[string[]]$groups

)

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") -eq $null) {return}

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Management.RegisteredServers") -eq $null) {return}

$cmserver = New-Object Microsoft.SqlServer.Management.Smo.Server $server

$sqlconnection = $cmserver.ConnectionContext.SqlConnectionObject

try { $cmstore = new-object Microsoft.SqlServer.Management.RegisteredServers.RegisteredServersStore($sqlconnection)}

catch { throw "Cannot access Central Management Server" }

$servers = @()

if ($groups -ne $null) {

foreach ($group in $groups) {

$cms = $cmstore.ServerGroups["DatabaseEngineServerGroup"].ServerGroups[$group]

$servers += ($cms.GetDescendantRegisteredServers()).servername

}

} else {

$cms = $cmstore.ServerGroups["DatabaseEngineServerGroup"]

$servers = ($cms.GetDescendantRegisteredServers()).servername

}

return $servers

}

Function Get-SqlMaxMemory {

<#

.SYNOPSIS

Displays information relating to SQL Server Max Memory configuration settings. Works on SQL Server 2000-2014.

.DESCRIPTION

Inspired by Jonathan Kehayias’s post about SQL Server Max memory (<a href="http://bit.ly/sqlmemcalc">http://bit.ly/sqlmemcalc</a>), this script displays a SQL Server’s:

total memory, currently configured SQL max memory, and the calculated recommendation.

Jonathan notes that the formula used provides a *general recommendation* that doesn’t account for everything that may be going on in your specific environment.

.PARAMETER Servers

Allows you to specify a comma separated list of servers to query.

.PARAMETER ServersFromFile

Allows you to specify a list that’s been populated by a list of servers to query. The format is as follows

server1

server2

server3

.PARAMETER SqlCms

Reports on a list of servers populated by the specified SQL Server Central Management Server.

.PARAMETER SqlCmsGroups

This is a parameter that appears when SqlCms has been specified. It is populated by Server Groups within the given Central Management Server.

.NOTES

Author : Chrissy LeMaire

Requires:         PowerShell Version 3.0, SQL Server SMO, sysadmin access on SQL Servers

DateUpdated: 2015-May-21

.LINK

<a href="https://gallery.technet.microsoft.com/scriptcenter/Get-Set-SQL-Max-Memory-19147057">https://gallery.technet.microsoft.com/scriptcenter/Get-Set-SQL-Max-Memory-19147057</a>

.EXAMPLE

Get-SqlMaxMemory -SqlCms sqlcluster

Get Memory Settings for all servers within the SQL Server Central Management Server "sqlcluster"

.EXAMPLE

Get-SqlMaxMemory -SqlCms sqlcluster | Where-Object { $_.SqlMaxMB -gt $_.TotalMB } | Set-SqlMaxMemory -UseRecommended

Find all servers in CMS that have Max SQL memory set to higher than the total memory of the server (think 2147483647)

#>

[CmdletBinding()]

Param(

[parameter(Position=0)]

[string[]]$Servers,

# File with one server per line

[string]$ServersFromFile,

# Central Management Server

[string]$SqlCms

)

DynamicParam { if ($SqlCms) { return (Get-ParamSqlCmsGroups $SqlCms) } }

PROCESS {

if ([string]::IsNullOrEmpty($SqlCms) -and [string]::IsNullOrEmpty($ServersFromFile) -and [string]::IsNullOrEmpty($servers))

{ throw "You must specify a server list source using -Servers or -SqlCms or -ServersFromFile" }

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") -eq $null )

{ throw "Quitting: SMO Required. You can download it from <a href="http://goo.gl/R4yA6u">http://goo.gl/R4yA6u</a>" }

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Management.RegisteredServers") -eq $null )

{ throw "Quitting: SMO Required. You can download it from <a href="http://goo.gl/R4yA6u">http://goo.gl/R4yA6u</a>" }

$SqlCmsGroups = $psboundparameters.SqlCmsGroups

if ($SqlCms) { $servers = Get-SqlCmsRegServers -server $SqlCms -groups $SqlCmsGroups }

If ($ServersFromFile) { $servers = Get-Content $ServersFromFile }

$collection = @()

foreach ($servername in $servers) {

Write-Verbose "Attempting to connect to $servername"

$server = New-Object Microsoft.SqlServer.Management.Smo.Server $servername

try { $server.ConnectionContext.Connect() } catch { Write-Warning "Can’t connect to $servername. Moving on."; continue }

$maxmem = $server.Configuration.MaxServerMemory.ConfigValue

$reserve = 1

$totalMemory = $server.PhysicalMemory

# Some servers underreport by 1MB.

if (($totalmemory % 1024) -ne 0) { $totalMemory = $totalMemory + 1 }

if ($totalMemory -ge 4096) {

$currentCount = $totalMemory

while ($currentCount/4096 -gt 0) {

if ($currentCount -gt 16384) {

$reserve += 1

$currentCount += -8192

} else {

$reserve += 1

$currentCount += -4096

}

}

}

$recommendedMax = [int]($totalMemory-($reserve*1024))

$object = New-Object PSObject -Property @{

Server = $server.name

TotalMB = $totalMemory

SqlMaxMB = $maxmem

RecommendedMB = $recommendedMax

}

$server.ConnectionContext.Disconnect()

$collection += $object

}

return ($collection | Sort-Object Server | Select Server, TotalMB, SqlMaxMB, RecommendedMB)

}

}

Function Set-SqlMaxMemory {

<#

.SYNOPSIS

Sets SQL Server max memory then displays information relating to SQL Server Max Memory configuration settings. Works on SQL Server 2000-2014.

.PARAMETER Servers

Allows you to specify a comma separated list of servers to query.

.PARAMETER ServersFromFile

Allows you to specify a list that’s been populated by a list of servers to query. The format is as follows

server1

server2

server3

.PARAMETER SqlCms

Reports on a list of servers populated by the specified SQL Server Central Management Server.

.PARAMETER SqlCmsGroups

This is a parameter that appears when SqlCms has been specified. It is populated by Server Groups within the given Central Management Server.

.PARAMETER MaxMB

Specifies the max megabytes

.PARAMETER UseRecommended

Inspired by Jonathan Kehayias’s post about SQL Server Max memory (<a href="http://bit.ly/sqlmemcalc">http://bit.ly/sqlmemcalc</a>), this uses a formula to determine the default optimum RAM to use, then sets the SQL max value to that number.

Jonathan notes that the formula used provides a *general recommendation* that doesn’t account for everything that may be going on in your specific environment.

.NOTES

Author : Chrissy LeMaire

Requires:         PowerShell Version 3.0, SQL Server SMO, sysadmin access on SQL Servers

DateUpdated: 2015-May-21

.LINK

<a href="https://gallery.technet.microsoft.com/scriptcenter/Get-Set-SQL-Max-Memory-19147057">https://gallery.technet.microsoft.com/scriptcenter/Get-Set-SQL-Max-Memory-19147057</a>

.EXAMPLE

Set-SqlMaxMemory sqlserver 2048

Set max memory to 2048 MB on just one server, "sqlserver"

.EXAMPLE

Get-SqlMaxMemory -SqlCms sqlcluster | Where-Object { $_.SqlMaxMB -gt $_.TotalMB } | Set-SqlMaxMemory -UseRecommended

Find all servers in CMS that have Max SQL memory set to higher than the total memory of the server (think 2147483647),

then pipe those to Set-SqlMaxMemory and use the default recommendation

.EXAMPLE

Set-SqlMaxMemory -SqlCms sqlcluster -SqlCmsGroups Express -MaxMB 512 -Verbose

Specifically set memory to 512 MB for all servers within the "Express" server group on CMS "sqlcluster"

#>

[CmdletBinding()]

Param(

[parameter(Position=0)]

[string[]]$Servers,

[parameter(Position=1)]

[int]$MaxMB,

[string]$ServersFromFile,

[string]$SqlCms,

[switch]$UseRecommended,

[Parameter(ValueFromPipeline=$True)]

[object]$collection

)

DynamicParam { if ($SqlCms) { return (Get-ParamSqlCmsGroups $SqlCms)} }

PROCESS {

if ([string]::IsNullOrEmpty($SqlCms) -and [string]::IsNullOrEmpty($ServersFromFile) -and [string]::IsNullOrEmpty($servers) -and $collection -eq $null)

{ throw "You must specify a server list source using -Servers or -SqlCms or -ServersFromFile or you can pipe results from Get-SqlMaxMemory" }

if ($MaxMB -eq 0 -and $UseRecommended -eq $false -and $collection -eq $null) { throw "You must specify -MaxMB or -UseRecommended" }

if ($collection -eq $null) {

$SqlCmsGroups = $psboundparameters.SqlCmsGroups

if ($SqlCmsGroups -ne $null) {

$collection = Get-SqlMaxMemory -Servers $servers -SqlCms $SqlCms -ServersFromFile $ServersFromFile -SqlCmsGroups $SqlCmsGroups

} else { $collection = Get-SqlMaxMemory -Servers $servers -SqlCms $SqlCms -ServersFromFile $ServersFromFile }

}

$collection | Add-Member -NotePropertyName OldMaxValue -NotePropertyValue 0

foreach ($row in $collection) {

$server = New-Object Microsoft.SqlServer.Management.Smo.Server $row.server

try { $server.ConnectionContext.Connect() } catch { Write-Warning "Can’t connect to $servername. Moving on."; continue }

if (!(Test-SqlSa $server)) {

Write-Warning "Not a sysadmin on $servername. Moving on."

$server.ConnectionContext.Disconnect()

continue

}

$row.OldMaxValue = $row.SqlMaxMB

try {

if ($UseRecommended) {

Write-Verbose "Changing $($row.server) SQL Server max from $($row.SqlMaxMB) to $($row.RecommendedMB) MB"

$server.Configuration.MaxServerMemory.ConfigValue = $row.RecommendedMB

$row.SqlMaxMB = $row.RecommendedMB

} else {

Write-Verbose "Changing $($row.server) SQL Server max from $($row.SqlMaxMB) to $MaxMB MB"

$server.Configuration.MaxServerMemory.ConfigValue = $MaxMB

$row.SqlMaxMB = $MaxMB

}

$server.Configuration.Alter()

} catch { Write-Warning "Could not modify Max Server Memory for $($row.server)" }

$server.ConnectionContext.Disconnect()

}

return $collection | Select Server, TotalMB, OldMaxValue, @{name="CurrentMaxValue";expression={$_.SqlMaxMB}}

}

}

Set-SqlMaxMemory $name 10000[/code]

Install Developer APPS

[code language=”powershell”]start-process F:\tools\7zipInstall.msi -ArgumentList "/q" -Wait

start-process F:\tools\ccleaner.exe -argumentlist "/S" -Wait

start-process F:\tools\fiddler4setup.exe -ArgumentList "/S" -Wait

start-process ‘F:\tools\Firefox Setup Stub 36.0.4.exe’ -ArgumentList "/S" -Wait

start-process F:\tools\iview438_setup.exe -ArgumentList "/silent" -Wait

start-process F:\tools\LINQPad4Setup.exe -ArgumentList "/silent" -Wait

start-process F:\tools\npp.6.7.5.Installer.exe -ArgumentList "/S" -Wait

start-process F:\tools\paint.net.4.0.5.install.exe -ArgumentList "/S" -Wait

start-process F:\tools\paint.net.4.0.5.install.exe -ArgumentList "/auto" -Wait

start-process F:\tools\PowerGUI.3.8.0.129.msi -ArgumentList "/q"[/code]

#start print spooler for cutepdf

[code language=”powershell”]net start spooler

sc query spooler[/code]

#fix hanging http://d4rkcell.com/archives/1217

[code language=”powershell”]start-process F:\tools\cutepdf-writer.exe -ArgumentList "/VERYSILENT /SUPPRESSMSGBOXES /NORESTART /SP-"

start-process F:\tools\VisualStudio2012\vs_premium.exe -ArgumentList "/adminfile AdminDeployment.xml /passive /norestart" -Wait -NoNewWindow -PassThru

$vsixInstallerPath = "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\VSIXInstaller.exe"

$extensionPath = "F:\tools\CKS.Dev11.vsix"

Start-Process -FilePath $vsixInstallerPath -ArgumentList "/q $extensionPath" -NoNewWindow -Wait

start-process F:\tools\SPDesigner\setup.exe -ArgumentList "/adminfile updates\adminfile.msp" -wait

start-process F:\tools\en_visual_studio_enterprise_2015_with_update_1_x86_x64_web_installer_8234346.exe -ArgumentList "/S /AdminFile F:\Tools\AdminDeployment.xml" -Wait[/code]

RDP to developer VM as svc_SPSetup and Launch AutoSPInstaller bat file

[code language=”powershell”]start-process F:\tools\AutoSPInstaller\SP\AutoSPInstaller\AutoSPInstallerLaunch.bat -wait[/code]

#Run autospinstaller

#Pre-reqs- 10 minutes w Restart

#Install binaries- 15 minutes

#Automation Fix- CU prompts for internet trusted file. 30 minutes (or right click all 3 CU files and go to Security and unblock, already done for June 2015 CU on AzureShare.).


#UPS Sync- we have to do this manually per install guide

#Add developer as full control of web applications

SharePoint- Configure User Profile AD Sync by hand

AD Connection:

Active Directory Company

Company.local

Company\svc_spups

passw0rd

(Sync All OUs)


Enable timerjob 1am daily:


Start full sync:


Manually add developer as full control of web applications.

Done!