Posted by: Eric Schrader | January 18, 2017

Azure Linux Ubuntu disk space full

I noticed our dev team had some issues with our disk space on an Azure VM saying the disk was full. I saw something in Linux called dev/sba1 that was taking up all my space and my disk was full. Why was my Linux storage space low?

Our website is only 5-6GB in size, so I knew something was wrong. Our VM in Azure is a DS12v2 with 56GB ram and 128GB SSD.

We are running the Ubuntu 14 OS image from Azure and are using it as an Apache web host.

When I ran a “df” (Disk Filesystem) command to check the free space, one of the volumes was huge and at 97%

$ sudo df -h

Filesystem Size Used Avail Use% Mounted on

udev 28G 12K 28G 1% /dev

tmpfs 5.6G 432K 5.6G 1% /run

/dev/sda1 29G 27G 1.1G 97% /

none 4.0K 0 4.0K 0% /sys/fs/cgroup

none 5.0M 0 5.0M 0% /run/lock

none 28G 0 28G 0% /run/shm

none 100M 0 100M 0% /run/user

none 64K 0 64K 0% /etc/network/interfaces.dynamic.d

/dev/sdb1 111G 60M 105G 1% /mnt

 

Run a sudo df -h (h stands for human readable)

I am still not 100% clear on this, but some of these above results are different disk partitions that are mounted via symbolic names, similar to how a disk in Windows can be partitioned into C, D drives, etc.

The Azure VM had one OS VHD assigned to it, which should be 128GB.

So, I focused in on the /dev/sda1 filesystem. I had no clue what this was at first, but after looking into it, it might be my VHD mounted to my VM’s primary root drive (/). (Please correct me if I am wrong). In “sda1“, The “sd” stands for SCSI device (which is now any attached device, could be USB, SATA, IDE, etc.), the “a” stands for the attached device order (a is first, b is the second device, etc.) and the “1” indicates the partition on that device (think of a hard drive partitioned into 1, 2, 3 different partitions.) (thanks to this article for the explanation http://superuser.com/questions/558156/what-does-dev-sda-for-linux-mean)

For me, I only have one sd device and one partition, so I assume that’s my Azure VM OS VHD that should have been 128GB. But why was it only 29GB?

WELL! All Linux OS vm partitions come as 30GB allocated.

How do I get all my GB’s? Add a second drive for my data? No, just resize the primary partition.

I read this article (https://blogs.msdn.microsoft.com/cloud_solution_architect/2016/05/24/step-by-step-how-to-resize-a-linux-vm-os-disk-in-azure-arm/) about resizing an Azure VHD and thought “ooh God, I am going to scrap my entire OS partitions data if this goes wrong or if I get stuck in these steps…” but after reading the top, UBUNTU automatically resizes the Linux partition on boot! YES! All I have to do is reboot! But, I just rebooted and that didn’t resize the partition….

The Problem: When browsing the Azure portal, I noticed my VM disk size was blank, and where I could select 128/256 or 512GB, none were selected. So, I thought “maybe Azure doesn’t automatically define a default OS disk size of 128GB since machine sizes can go up or down dynamically.”

These machines will have 128GB OS disks allocated to them, so I wanted to set them to the full 128GB (I can go up in size, but not down).

Problem: my OS disk size is not selected, so Ubuntu cannot automatically resize the partition (I think the VHD is dynamically allocated at this point)

Solution: How to add more space in the Azure Portal easily with a Linux Ubuntu VM

  1. Turn off the VM
  2. Select the disk size for the OS disk (I used 128GB)
  3. Turn it on.

BAM! You now have more space.

Run the “df -h” command again after the VM comes back online and see a 126GB of space at the root! Done!

$ sudo df -h

Filesystem Size Used Avail Use% Mounted on

udev 28G 12K 28G 1% /dev

tmpfs 5.6G 416K 5.6G 1% /run

/dev/sda1 126G 5.8G 116G 5% /

none 4.0K 0 4.0K 0% /sys/fs/cgroup

none 5.0M 0 5.0M 0% /run/lock

none 28G 0 28G 0% /run/shm

none 100M 0 100M 0% /run/user

none 64K 0 64K 0% /etc/network/interfaces.dynamic.d

/dev/sdb1 111G 60M 105G 1% /mnt

 

If you don’t have Ubuntu, you have more steps to do to resize the Linux OS partition. I haven’t done it, but this seems like a good place to start: https://blogs.msdn.microsoft.com/cloud_solution_architect/2016/05/24/step-by-step-how-to-resize-a-linux-vm-os-disk-in-azure-arm/

 

Please leave any comments if you know more about the df command results, the sda1, Azure VM OS disk sizes, Linux partitions, etc. I am always learning.

References that helped me get here:

 

Posted by: Eric Schrader | December 9, 2016

Search Result preview images in SharePoint Online

SharePoint search results OOTB do not display image previews until you hover. We wanted to have a baseball card type approach to display certain items.

Here is an unfinished example of the results displayed as cards with image previews and the OOTB hover panel:

  1. Make sure the site is a publishing portal and publishing features are enabled at the site collection and site level. In order to get the Search Display templates to display the *.html files in the masterpage gallery, some of these features have to be on. Otherwise you just have *.JS files which can limit you. Just ask me under comments if you have any questions.
  2. Modify your Search display template for result items:
    1. /_catalogs/masterpage/Forms/Display%20Templates.aspx
    2. I modified Item_CommonItem_Body.html.
  3. Edit Item_CommonItem_Body.html properties and add the following property:
    1. ServerRedirectedPreviewURL
    2. (properties are separated by a comma, and surrounded by single quotes. So the exact text I added to the end was ,’ ServerRedirectedPreviewURL’ (but replace my blog posts fancy quotes)
  4. Edit the Item_CommonItem_Body.html file (I open with Explorer and edit the file in notepad++)
    1. Verify the property was added:

    2. Next, add a JavaScript tag to store the Preview Image URL as a web safe string:

      imageurlpreview = $htmlEncode(ctx.CurrentItem.ServerRedirectedPreviewURL)

    3. Now, let’s add in our custom HTML. This is kinda “hacky” since I am using this site for a proof of concept and I don’t care if these customizations exist everywhere in my test site collection. Ask me below in comments if you want to know how to copy this file and isolate the results to use this custom template using result types or a custom search results page.
        1. I scrolled down to the first HTML div I found, “ms-srch-item-body”. Right above this, I added my custom Baseball Card HTML. Then I moved the rest of this stuff in except for the closing div tag.
        2. The main thing was this line to add the new JavaScript for the image:

          <img src=”_#= imageurlpreview =#_” alt=”Preview” />

          Here is my baseball card HTML (including the preview image)

          
          <div class="cbs-PictureCardsContainer">
          
          <div class="cbs-PictureCardsImageContainer">
           <a title="Title here" class="cbs-pictureImgLink" href="#">
          
          <div class="cbs-PictureCardsImage">
           <img src="_#= imageurlpreview =#_" alt="Preview" />
           </div>
          
          
          <div class="sts-cardtype sts-cardtypeidea">
           Type</div>
          
           </a></div>
          
          
          <div title="" class="cbs-pictureCardsCategory ms-noWrap">
           Category</div>
          
          
          <div class="cbs-PictureCardsDataContainer">
           <a title="Title here" class="cbs-PictureCardsLine1Link" href="#">
          
          <div class="cbs-PictureCardsTitle ms-noWrap">
           Pic title</div>
          
           </a>
           
           
           Move all of the OOTB stuff here, starting with the div ms-srch-item-body
          
           
           
          
          <div title="" class="cbs-PictureCardsDesc">
           Description of image</div>
          
           
           </div>
          
           </div>
          
          
        3. Now Save the display template HTML file and publish ONLY the HTML file from the browser (the JS file gets automatically updated instantly):

      Add the baseball card CSS to your result page and you should be good. Again, the element selector (ms-srch-item) for floating these elements is a bit hokey, I could have modified the Control_SearchResults and individual Item templates but this is just a POC.
      /* Cards */
      .ms-srch-item {
      width:240px;
      display:inline;
      float:left;
      margin:11px;
      border: 1px solid #DDD;
      clear:none;
      }
      .cbs-PictureCardsImageContainer{
      height:240px;
      border-top-left-radius: 9px;
      border-top-right-radius: 9px;
      width:240px;
      
      }
      .cbs-PictureCardsImage {
      height:240px;
      overflow:hidden;
      width:240px;
      
      }
      .cbs-noImageContainer-ContentWrapperLarge {
      display:none;
      }
      .cbs-PictureCardsDataContainer {
      padding: 8px 22px 0px 22px;
      background-color: #f8f8f8;
      color: #212121;
      }
      .cbs-PictureCardsDataContainer a, .cbs-PictureCardsDataContainer a:active, .cbs-PictureCardsDataContainer a:hover, .cbs-PictureCardsDataContainer a:visited {
      color: #212121;
      }
      .cbs-pictureCardsCategory {
      background-color:#666;
      color: #FFF;
      font-weight: bold;
      font-size: 12px;
      padding: 8px 22px 8px 22px;
      border-top:1px solid #FFF;
      }
      .cbs-PictureCardsTitle {
      font-weight: bold;
      }
      .cbs-PictureCardsDesc {
      height: 75px;
      overflow:hidden;
      
      }
      
      .sts-cardtype {
      position:relative;
      top:-18px;
      left:120px;
      text-align:center;
      height: 18px;
      width: 120px;
      color:#000;
      font-weight:bold;
      }
      
      .sts-cardtypeidea {
      background-color:#a8da69;
      
      }
      
      .cbs-PictureCardsIconSection {
      float:left;
      margin-top:8px;
      margin-bottom:8px;
      }
      
    4. Check in and publish your custom result page (the custom ASPX page with all of your search refiners, result web part, search box, above CSS, etc.) and you should be good.

I have been searching for a way to rapidly create standalone Developer SharePoint 2013 standalone VM’s joined to a central domain for our in-house developers. Our team has created 60+ SharePoint VMs on Azure and continue to create about 10 per month. We are beginning to treat our VMs needing hours of repair like cattle, and no longer like the family dog if they have issues or “get terminally ill”, VM’s are replaced with a brand new shiny cow within 4 hoursJ

The process of manually creating VMs was not fun, taking over a day per VM in the beginning on average. I did not want to use Sysprep, since I would have to maintain multiple VM Images at a single point in time. For some, this might be the best way to go.

My solution was to create a lightweight PowerShell set of scripts that create the VM in Azure, install the applications, and keep everything consistent. I can create 4+ VMs at one time, all under ~4 hours total. This is a process that scales somewhat, to meet our needs. Perfect.

Alternatives to this manual PowerShell process I went through:

  • Sysprep would save a ton of steps, but is not as easy to update OS patches, etc. as newer software comes out
  • AzureRM– Azure Resource Manager is a lot easier. However, these topologies seem somewhat isolated and all of our existing VMs and didn’t work well with what we had in place for our network/VPN, etc.
  • Azure SharePoint QuickStart templates/images- These were preconfigured and had various OS settings changed. Similar to the above Azure RM solution issues we ran into.

Assumptions:

  • You have an Azure subscription all set up, with a virtual network/DNS/Subnet (we have a site-to-site VPN)
  • You have domain controller with all of the SharePoint service accounts created for Least Privileges security installation
  • You have installed the latest version of Azure PowerShell installed, rebooted after installing it, and performed the Add-AzureAccount command
  • You have used AutoSPInstaller before

What you need:

  • The following information from Azure
    • Subscription ID
    • Virtual Network info
      • Network Name
      • Subnet Name
    • Resource Group
  • The following pre-existing VMs:
    • DC
      • Service Accounts
    • Fileshare VM
      • All of the necessary ISO’s and EXE’s
        • Installer files:
          • 7zipInstall.msi
          • ccleaner.exe
          • fiddler4setup.exe
          • Copy-Item ‘C:\Fileshare\applications\Firefox Setup Stub 36.0.4.exe’
          • iview438_setup.exe
          • LINQPad4Setup.exe
          • npp.6.7.5.Installer.exe
          • paint.net.4.0.5.install.exe
          • PowerGUI.3.8.0.129.msi
          • cutepdf-writer.exe
          • CKS.Dev11.vsix
          • codecompare.exe
      • Stand Alone EXE’s
        • ULSViewer.zip
        • U2U.SharePoint.CQB2010.zip
      • Applications (Extracted into their own folder with configuration.ini files)
        • CamlDesigner2013
        • Visual Studio 2012
        • Visual Studio 2015
        • SharePoint Designer 2013
        • en_sql_server_2014_enterprise_edition_with_service_pack_1_x64_dvd_6669618
        • AutoSPInstaller for dev
        • sql2014config file for dev
        • en_sharepoint_server_2013_with_sp1_x64_dvd_3823428
        • SharePoint 2013 June 2015 CU (note, if you download this from the internet, uncheck the security property so you don’t get prompted during the AutoSPInstaller process for UAC- right click all 3 CU files and go to Security and unblock, you only have to do this one time on the fileshare.)
  • Silent install for software (one-time prep, then save the folder on the Fileshare VM)

Configure PowerShell variables for the new standalone developer SharePoint VM

#VM Name will be ASP13D08

#IP will be 192.168.1.87

#Cloud service Company-Redondo-D08 (each developer has their own cloud service so they can power on VMs without having to wait for the other developers to start at the same time)

#service accounts

#Single SharePoint 2013 developer VM

$varVMLocation = "A"
$varVMServerType = "SP"
$varVMSPVersion = "13"
$varVMType = "D"
$varVMIntanceNum = "08"
$varVMReduxSuffix = "" # I sometimes append a version letter to the end of the developers VM, if they are getting an additional VM of the same role.
$spsetupname = "svc_spsetup"
$spsetuppasstext = "passw0rdspsetup"
$users = @("svc_spsetup", "svc_spfarm", "eric.schrader", "dev1", "etc");
$varVMStaticIP = "192.168.1.87"
$varStorageAccount = "Company" + $varVMLocation + $varVMType + $varVMIntanceNum #unique
$varStorageAccount = $varStorageAccount.ToLower()
$service = "Company-Redondo-" + $varVMType + $varVMIntanceNum
$instancesize = "Basic_A4"
$subscriptionid = "12345678-12345-123456"
$subscriptionName = "Microsoft Azure Enterprise"
$imageFamily = "Windows Server 2012 R2 Datacenter" #Azure VM Image name, the latest will be used below.
$localadminname = "company.admin" #cant be "administrator", etc.
$localadminPassword = "passw0rdlocaladmin"
$joindomain = "Domain.local"
$domainname = "Domain"
$machineOU = "OU=Azure,OU=Development,OU=Servers,OU=Seattle,DC=Company,DC=local"
$timezone = "Pacific Standard Time"
$domainusername = "svc_spsetup"
$domainpassword = "passw0rdspsetup"
$datadiskGB = 127
$datadiskLUN = 0
$datadiskCACHE = "None"
$vmsubnet = "Subnet-1"
$vmaffinitygroup = "VPN-Linked"

Create the Azure storage account if it doesn’t exist, the set it as the default for PowerShell

#Get-AzureStorageAccount | ft

#Change varStorageAccount to lowercase

$lowerStorageAccount = $varStorageAccount.ToLower()

Try{

get-azurestorageaccount -storageaccountname $varStorageAccount -ErrorAction Stop

#if this fails to get it, it will create it below. Need above error action

}

Catch {

#you got an error trying to get it, so create it.

Write-output "creating storage account $varStorageAccount"

New-AzureStorageAccount -StorageAccountName $lowerStorageAccount -Label $lowerStorageAccount -AffinityGroup $vmaffinitygroup

}

#now that it exists, set it as default.

Set-AzureSubscription -CurrentStorageAccountName $varStorageAccount -SubscriptionId $subscriptionid

Select-AzureSubscription -SubscriptionId $subscriptionid -Current

Create the VM using above variables

#try to fix the DNS error in the Comapny DC, WARNING: The specified DNS name is already taken.

#New-AzureService -Label $service -Description $service -AffinityGroup $vmaffinitygroup -ServiceName $service

 

</span>New-AzureVMConfig -Name $name -InstanceSize $instancesize -ImageName $image | Add-AzureProvisioningConfig -AdminUserName $localadminname -EnableWinRMHttp -TimeZone $timezone -DisableAutomaticUpdates –Password $localadminPassword -WindowsDomain -JoinDomain $joindomain -Domain $domainname -DomainUserName $domainusername -DomainPassword $domainpassword -MachineObjectOU $machineOU | Add-AzureDataDisk -CreateNew -DiskSizeInGB $datadiskGB -DiskLabel $datadiskname -LUN $datadiskLUN -HostCaching $datadiskCACHE | Set-AzureSubnet –SubnetNames $vmsubnet | Set-AzureStaticVNetIP -IPAddress $varVMStaticIP| New-AzureVM –ServiceName $service -AffinityGroup $vmaffinitygroup

Configures Secure Remote PowerShell Access to Windows Azure Virtual Machines

Download PS1 file from this blog post to your local computer with Azure PowerShell. https://gallery.technet.microsoft.com/scriptcenter/Configures-Secure-Remote-b137f2fe

#CD to location in PowerShell, for example, your desktop:

Cd C:\users\eric.schrader\desktop

#Create WInRM Cert to new VM

.\InstallWinRMCertAzureVM.ps1 -SubscriptionName $subscriptionname -ServiceName $service -Name $name

Connect to remote session

#Connect via remote powershell as local azure.admin
#uses variables from when the VM was created above

$passwordsec = convertto-securestring $localadminPassword -asplaintext -force

$user = $name +"\"+ $localadminname

$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $user,$passwordsec

$uri = Get-AzureWinRMUri -ServiceName $service -Name $name

Enter-PSSession -ConnectionUri $uri -Credential $cred

$env:computername

#Run variables Again!!!

# IMPORTANT, COPY AND PASTE THE ABOVE VARIABLES SECTION IN AGAIN. This is a new remote session to the new Azure VM.

$varVMLocation = "A"

$varVMServerType = "SP"

$varVMSPVersion = "13"

$varVMType = "D"

$varVMIntanceNum = "08"

$varVMReduxSuffix = "" # I sometimes append a version letter to the end of the developers VM, if they are getting an additional VM of the same role.

$spsetupname = "svc_spsetup"

$spsetuppasstext = "passw0rdspsetup"

$users = @("svc_spsetup", "svc_spfarm", "eric.schrader", "dev1", "etc");

$varVMStaticIP = "192.168.1.87"

$varStorageAccount = "Company" + $varVMLocation + $varVMType + $varVMIntanceNum #unique

$varStorageAccount = $varStorageAccount.ToLower()

$service = "Company-Redondo-" + $varVMType + $varVMIntanceNum

$instancesize = "Basic_A4"

$subscriptionid = "12345678-12345-123456"

$subscriptionName = "Microsoft Azure Enterprise"

$imageFamily = "Windows Server 2012 R2 Datacenter" #Azure VM Image name, the latest will be used below.

$localadminname = "company.admin" #cant be "administrator", etc.

$localadminPassword = "passw0rdlocaladmin"

$joindomain = "Domain.local"

$domainname = "Domain"

$machineOU = "OU=Azure,OU=Development,OU=Servers,OU=Seattle,DC=Company,DC=local"

$timezone = "Pacific Standard Time"

$domainusername = "svc_spsetup"

$domainpassword = "passw0rdspsetup"

$datadiskGB = 127

$datadiskLUN = 0

$datadiskCACHE = "None"

$vmsubnet = "Subnet-1"

$vmaffinitygroup = "VPN-Linked"

Set proper storage account in remote session

# Now that you set the variables, set the storage account for the remote session

Set-AzureSubscription -CurrentStorageAccountName $varStorageAccount -SubscriptionId $subscriptionid

Select-AzureSubscription -SubscriptionId $subscriptionid -Current

Format F drive for SharePoint/SQL, permission service accounts

#Format F drive

$labels = @("DATA1","DATA2")

Write-Host "Initializing and formatting raw disks"

$disks = Get-Disk | Where partitionstyle -eq 'raw' | sort number

## start at F: because sometimes E: shows up as a CD drive in Azure

$letters = 70..89 | ForEach-Object { ([char]$_) }

$count = 0

foreach($d in $disks) {

$driveLetter = $letters[$count].ToString()

$d |

Initialize-Disk -PartitionStyle MBR -PassThru |

New-Partition -UseMaximumSize -DriveLetter $driveLetter |

Format-Volume -FileSystem NTFS -NewFileSystemLabel $labels[$count] `

-Confirm:$false -Force

$count++

}

GET-WMIOBJECT –query "SELECT * from win32_logicaldisk where DriveType = '3'"

#add developer, and admins/spsetup/spfarm, set in $users variable above

foreach($user in $users) {

$domainuser= $domainname + "\"+$user

$Group = "Administrators"

$de = [ADSI]"WinNT://$name/$Group,group"

$de.Add("WinNT://$domainname/$user")

Write-Host "Done, $domainuser has been permissioned to this computer."

}

net localgroup administrators

#create folder and share for apps

New-Item -Path F:\tools -ItemType directory -Value Tools

New-SMBShare –Name "Tools" –Path "F:\Tools" -ChangeAccess "Everyone"

 Disable UAC (for developers), restart computer, set execution policy, etc.

#Disable UAC

Set-ItemProperty -Path HKLM:\Software\Microsoft\Windows\CurrentVersion\policies\system -Name EnableLUA -Value 0

#allow scripts

Set-executionpolicy unrestricted -force

#reboot

Restart-computer

#wait 5 minutes for reboot

#Reconnect to powershell

(exit, reconnect to remote powershell, re-run vars)

 Connect to remote session

#Connect via remote powershell as local azure.admin
#uses variables from when the VM was created above

$passwordsec = convertto-securestring $localadminPassword -asplaintext -force

$user = $name +"\"+ $localadminname

$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $user,$passwordsec

$uri = Get-AzureWinRMUri -ServiceName $service -Name $name

Enter-PSSession -ConnectionUri $uri -Credential $cred

$env:computername

#Run variables Again!!!

# IMPORTANT, COPY AND PASTE THE ABOVE VARIABLES SECTION IN AGAIN. This is a new remote session to the new Azure VM.

 

</span>$varVMLocation = "A"

$varVMServerType = "SP"

$varVMSPVersion = "13"

$varVMType = "D"

$varVMIntanceNum = "08"

$varVMReduxSuffix = "" # I sometimes append a version letter to the end of the developers VM, if they are getting an additional VM of the same role.

$spsetupname = "svc_spsetup"

$spsetuppasstext = "passw0rdspsetup"

$users = @("svc_spsetup", "svc_spfarm", "eric.schrader", "dev1", "etc");

$varVMStaticIP = "192.168.1.87"

$varStorageAccount = "Company" + $varVMLocation + $varVMType + $varVMIntanceNum #unique

$varStorageAccount = $varStorageAccount.ToLower()

$service = "Company-Redondo-" + $varVMType + $varVMIntanceNum

$instancesize = "Basic_A4"

$subscriptionid = "12345678-12345-123456"

$subscriptionName = "Microsoft Azure Enterprise"

$imageFamily = "Windows Server 2012 R2 Datacenter" #Azure VM Image name, the latest will be used below.

$localadminname = "company.admin" #cant be "administrator", etc.

$localadminPassword = "passw0rdlocaladmin"

$joindomain = "Domain.local"

$domainname = "Domain"

$machineOU = "OU=Azure,OU=Development,OU=Servers,OU=Seattle,DC=Company,DC=local"

$timezone = "Pacific Standard Time"

$domainusername = "svc_spsetup"

$domainpassword = "passw0rdspsetup"

$datadiskGB = 127

$datadiskLUN = 0

$datadiskCACHE = "None"

$vmsubnet = "Subnet-1"

$vmaffinitygroup = "VPN-Linked"

Install SQL pre-reqs

 #Install .net 3.5 for SQL prereq on SQL server

Install-WindowsFeature –name NET-Framework-Core

Now that the VM is ready for software installs, lets copy the software over. Due to the Windows “triple hop” issue of credentials, I cannot remote into the VM then copy from a 3rd remote location to the vm. I will have to RDP manually

RDP to fileshare computer as svc_SPSetup

Run variables on fileshare computers PowerShell

#Run variables Again!!!

# IMPORTANT, COPY AND PASTE THE ABOVE VARIABLES SECTION IN AGAIN. This is a new VM session.

$varVMLocation = "A"

$varVMServerType = "SP"

$varVMSPVersion = "13"

$varVMType = "D"

$varVMIntanceNum = "08"

$varVMReduxSuffix = "" # I sometimes append a version letter to the end of the developers VM, if they are getting an additional VM of the same role.

$spsetupname = "svc_spsetup"

$spsetuppasstext = "passw0rdspsetup"

$users = @("svc_spsetup", "svc_spfarm", "eric.schrader", "dev1", "etc");

$varVMStaticIP = "192.168.1.87"

$varStorageAccount = "Company" + $varVMLocation + $varVMType + $varVMIntanceNum #unique

$varStorageAccount = $varStorageAccount.ToLower()

$service = "Company-Redondo-" + $varVMType + $varVMIntanceNum

$instancesize = "Basic_A4"

$subscriptionid = "12345678-12345-123456"

$subscriptionName = "Microsoft Azure Enterprise"

$imageFamily = "Windows Server 2012 R2 Datacenter" #Azure VM Image name, the latest will be used below.

$localadminname = "company.admin" #cant be "administrator", etc.

$localadminPassword = "passw0rdlocaladmin"

$joindomain = "Domain.local"

$domainname = "Domain"

$machineOU = "OU=Azure,OU=Development,OU=Servers,OU=Seattle,DC=Company,DC=local"

$timezone = "Pacific Standard Time"

$domainusername = "svc_spsetup"

$domainpassword = "passw0rdspsetup"

$datadiskGB = 127

$datadiskLUN = 0

$datadiskCACHE = "None"

$vmsubnet = "Subnet-1"

$vmaffinitygroup = "VPN-Linked"

Copy the software

#Run from Fileshare as SPSetup in PowerShell

#re-run variables

#run installers
#Copy applications from local computer S drive on \\fileshare to server F drive

Copy-Item C:\Fileshare\applications\7zipInstall.msi -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\ccleaner.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\fiddler4setup.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item 'C:\Fileshare\applications\Firefox Setup Stub 36.0.4.exe' -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\iview438_setup.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\LINQPad4Setup.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\npp.6.7.5.Installer.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\paint.net.4.0.5.install.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\PowerGUI.3.8.0.129.msi -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\cutepdf-writer.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\CKS.Dev11.vsix -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\codecompare.exe -Destination <a href="///\\$name\tools">\\$name\tools</a>

#copy exes to F:\Tools 

Copy-Item C:\Fileshare\applications\ULSViewer.zip -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\U2U.SharePoint.CQB2010.zip -Destination <a href="///\\$name\tools">\\$name\tools</a>

Copy-Item C:\Fileshare\applications\CamlDesigner2013\* -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

Copy-Item "C:\Fileshare\Visual Studio 2012\*" -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

Copy-Item "C:\Fileshare\Visual Studio 2015\*" -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

Copy-Item "C:\Fileshare\SharePoint Designer 2013\*" -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

#copy SQL to SQL server (Copy ISO CONTENTS)

Copy-Item C:\Fileshare\en_sql_server_2014_enterprise_edition_with_service_pack_1_x64_dvd_6669618\* -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

Copy-Item C:\Fileshare\AutoSPInstallerDev2013\* -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse

Copy-Item C:\Fileshare\sql2014configdevint\* -Destination <a href="///\\$name\tools">\\$name\tools</a> -Recurse -force

Copy-Item C:\Fileshare\en_sharepoint_server_2013_with_sp1_x64_dvd_3823428\* -Destination <a href="///\\$name\tools\AutoSPInstaller\SP\2013\SharePoint">\\$name\tools\AutoSPInstaller\SP\2013\SharePoint</a> -Recurse -force

Copy-Item "C:\Fileshare\SharePoint 2013 June 2015 CU" -Destination <a href="///\\$name\tools\AutoSPInstaller\SP\2013\Updates">\\$name\tools\AutoSPInstaller\SP\2013\Updates</a> -Recurse -force

Close RDP to fileshare and go back to your local computers PowerShell. We are ready to install the software

#re-run variables

#Run variables Again!!!

# IMPORTANT, COPY AND PASTE THE ABOVE VARIABLES SECTION IN AGAIN. This is a new remote session to the new Azure VM.

$varVMLocation = "A"

$varVMServerType = "SP"

$varVMSPVersion = "13"

$varVMType = "D"

$varVMIntanceNum = "08"

$varVMReduxSuffix = "" # I sometimes append a version letter to the end of the developers VM, if they are getting an additional VM of the same role.

$spsetupname = "svc_spsetup"

$spsetuppasstext = "passw0rdspsetup"

$users = @("svc_spsetup", "svc_spfarm", "eric.schrader", "dev1", "etc");

$varVMStaticIP = "192.168.1.87"

$varStorageAccount = "Company" + $varVMLocation + $varVMType + $varVMIntanceNum #unique

$varStorageAccount = $varStorageAccount.ToLower()

$service = "Company-Redondo-" + $varVMType + $varVMIntanceNum

$instancesize = "Basic_A4"

$subscriptionid = "12345678-12345-123456"

$subscriptionName = "Microsoft Azure Enterprise"

$imageFamily = "Windows Server 2012 R2 Datacenter" #Azure VM Image name, the latest will be used below.

$localadminname = "company.admin" #cant be "administrator", etc.

$localadminPassword = "passw0rdlocaladmin"

$joindomain = "Domain.local"

$domainname = "Domain"

$machineOU = "OU=Azure,OU=Development,OU=Servers,OU=Seattle,DC=Company,DC=local"

$timezone = "Pacific Standard Time"

$domainusername = "svc_spsetup"

$domainpassword = "passw0rdspsetup"

$datadiskGB = 127

$datadiskLUN = 0

$datadiskCACHE = "None"

$vmsubnet = "Subnet-1"

$vmaffinitygroup = "VPN-Linked"

RDP to Developer VM using svc_SPSetup and install SQL by PowerSHell.
#via SPSetup , possibly have to sign into RDP instead of remote PS. Takes 20 minutes

start-process F:\tools\sql\Setup.exe -ArgumentList "/q /SkipRules=VSShellInstalledRule RebootRequiredCheck /ConfigurationFile=F:\Tools\ConfigurationFile.ini /ERRORREPORTING=1 /IACCEPTSQLSERVERLICENSETERMS" -Wait

Close RDP to developer VM once SQL is done.

From local computer, Connect via remote powershell as spsetup. There is code in here to install SQL remotely, but it too has a triple hop credential issue since I use service accounts. I just RDP to the VM and install SQL via PowerShell there.
#uses variables from when the VM was created above

$passwordsec = convertto-securestring $spsetuppasstext -asplaintext -force

$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $spsetupname ,$passwordsec

$uri = Get-AzureWinRMUri -ServiceName $service -Name $name

Enter-PSSession -ConnectionUri $uri -Credential $cred

$env:computername

#FIX – maybe to install SQL remotely
#Invoke-Command -ComputerName $name -Authentication CredSSP -credential $cred -scriptblock {
#F:\tools\sql\Setup.exe -ArgumentList “/q /SkipRules=VSShellInstalledRule RebootRequiredCheck /ConfigurationFile=F:\Tools\ConfigurationFile.ini /ERRORREPORTING=1 /IACCEPTSQLSERVERLICENSETERMS” -Wait
#}

Set SQL max memory to 10GB, set Max Degree of parallelism to 1 (this is a huge script, maybe you can shorten it)

#Set Max degree of parallelism to 1

## Sets the 'max degree of parallelism' value to 1 for the specified SQL server instance
## Port 1433 is used if not specified
## 2012-10-08
## <a href="http://www.pointbeyond.com">www.pointbeyond.com</a>
## NOTE: This function requires at least serveradmin level permissions within SQL server
function SetMaxDegreeOfParallelism()

{

Param(

$server,

$port="1433")

$conn = new-object System.Data.SqlClient.SqlConnection

try

{

$connectionString = "Server="+$server+","+$port+";Database=master;Integrated Security=True;"

$conn.ConnectionString = $connectionString

$conn.Open()

$cmd = new-object System.Data.SqlClient.SqlCommand

$cmd.Connection = $conn

# Ensure advanced options are available

$commandText = "sp_configure 'show advanced options', 1;RECONFIGURE WITH OVERRIDE;"

$cmd.CommandText = $commandText

$r = $cmd.ExecuteNonQuery()

# Set the Max Degree of Parallelism value to 1

write-host "Setting 'max degree of parallelism' value to 1 for server $server..."

$commandText = "sp_configure 'max degree of parallelism', 1;RECONFIGURE WITH OVERRIDE"

$cmd.CommandText = $commandText

$r = $cmd.ExecuteNonQuery()

write-host "Success"

}

catch

{

write-host "An error occurred trying to set the MaxDegreeOfParallelism value to 1 for server $server" -Fore Red

write-host "Ensure that server and port parameters are correct and that the current user has at least serveradmin permissions within SQL" -Fore Red

}

finally

{

$conn.Close()

}

}

# Call the function passing in SQL server name/instance/alias and port number

SetMaxDegreeOfParallelism -server $name -port "1433"

#Set SQL Max memory – 3GB

Function Test-SqlSa {

<#

.SYNOPSIS

Ensures sysadmin account access on SQL Server. $server is an SMO server object.

.EXAMPLE

if (!(Test-SQLSA $server)) { throw "Not a sysadmin on $source. Quitting." }

.OUTPUTS

$true if syadmin

$false if not

#>

[CmdletBinding()]

param(

[Parameter(Mandatory = $true)]

[ValidateNotNullOrEmpty()]

[object]$server

)

try {

return ($server.ConnectionContext.FixedServerRoles -match "SysAdmin")

}

catch { return $false }

}

Function Get-ParamSqlCmsGroups {

<#

.SYNOPSIS

Returns System.Management.Automation.RuntimeDefinedParameterDictionary

filled with server groups from specified SQL Server Central Management server name.

.EXAMPLE

Get-ParamSqlCmsGroups sqlserver

#>

[CmdletBinding()]

param(

[Parameter(Mandatory = $true)]

[string]$Server

)

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") -eq $null) {return}

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Management.RegisteredServers") -eq $null) {return}

$cmserver = New-Object Microsoft.SqlServer.Management.Smo.Server $server

$sqlconnection = $cmserver.ConnectionContext.SqlConnectionObject

try { $cmstore = new-object Microsoft.SqlServer.Management.RegisteredServers.RegisteredServersStore($sqlconnection)}

catch { return }

if ($cmstore -eq $null) { return }

$newparams = New-Object System.Management.Automation.RuntimeDefinedParameterDictionary

$paramattributes = New-Object System.Management.Automation.ParameterAttribute

$paramattributes.ParameterSetName = "__AllParameterSets"

$paramattributes.Mandatory = $false

$argumentlist = $cmstore.DatabaseEngineServerGroup.ServerGroups.name

if ($argumentlist -ne $null) {

$validationset = New-Object System.Management.Automation.ValidateSetAttribute -ArgumentList $argumentlist

$combinedattributes = New-Object -Type System.Collections.ObjectModel.Collection[System.Attribute]

$combinedattributes.Add($paramattributes)

$combinedattributes.Add($validationset)

$SqlCmsGroups = New-Object -Type System.Management.Automation.RuntimeDefinedParameter("SqlCmsGroups", [String[]], $combinedattributes)

$newparams.Add("SqlCmsGroups", $SqlCmsGroups)

return $newparams

} else { return }

}

Function Get-SqlCmsRegServers {

<#

.SYNOPSIS

Returns array of server names from CMS Server. If -Groups is specified,

only servers within the given groups are returned.

.EXAMPLE

Get-SqlCmsRegServers -Server sqlserver -Groups "Accounting", "HR"

#>

[CmdletBinding()]

param(

[Parameter(Mandatory = $true)]

[string]$server,

[string[]]$groups

)

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") -eq $null) {return}

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Management.RegisteredServers") -eq $null) {return}

$cmserver = New-Object Microsoft.SqlServer.Management.Smo.Server $server

$sqlconnection = $cmserver.ConnectionContext.SqlConnectionObject

try { $cmstore = new-object Microsoft.SqlServer.Management.RegisteredServers.RegisteredServersStore($sqlconnection)}

catch { throw "Cannot access Central Management Server" }

$servers = @()

if ($groups -ne $null) {

foreach ($group in $groups) {

$cms = $cmstore.ServerGroups["DatabaseEngineServerGroup"].ServerGroups[$group]

$servers += ($cms.GetDescendantRegisteredServers()).servername

}

} else {

$cms = $cmstore.ServerGroups["DatabaseEngineServerGroup"]

$servers = ($cms.GetDescendantRegisteredServers()).servername

}

return $servers

}

Function Get-SqlMaxMemory {

<#

.SYNOPSIS

Displays information relating to SQL Server Max Memory configuration settings. Works on SQL Server 2000-2014.

.DESCRIPTION

Inspired by Jonathan Kehayias's post about SQL Server Max memory (<a href="http://bit.ly/sqlmemcalc">http://bit.ly/sqlmemcalc</a>), this script displays a SQL Server's:

total memory, currently configured SQL max memory, and the calculated recommendation.

Jonathan notes that the formula used provides a *general recommendation* that doesn't account for everything that may be going on in your specific environment.

.PARAMETER Servers

Allows you to specify a comma separated list of servers to query.

.PARAMETER ServersFromFile

Allows you to specify a list that's been populated by a list of servers to query. The format is as follows

server1

server2

server3

.PARAMETER SqlCms

Reports on a list of servers populated by the specified SQL Server Central Management Server.

.PARAMETER SqlCmsGroups

This is a parameter that appears when SqlCms has been specified. It is populated by Server Groups within the given Central Management Server.

.NOTES

Author : Chrissy LeMaire

Requires:         PowerShell Version 3.0, SQL Server SMO, sysadmin access on SQL Servers

DateUpdated: 2015-May-21

.LINK

<a href="https://gallery.technet.microsoft.com/scriptcenter/Get-Set-SQL-Max-Memory-19147057">https://gallery.technet.microsoft.com/scriptcenter/Get-Set-SQL-Max-Memory-19147057</a>

.EXAMPLE

Get-SqlMaxMemory -SqlCms sqlcluster

Get Memory Settings for all servers within the SQL Server Central Management Server "sqlcluster"

.EXAMPLE

Get-SqlMaxMemory -SqlCms sqlcluster | Where-Object { $_.SqlMaxMB -gt $_.TotalMB } | Set-SqlMaxMemory -UseRecommended

Find all servers in CMS that have Max SQL memory set to higher than the total memory of the server (think 2147483647)

#>

[CmdletBinding()]

Param(

[parameter(Position=0)]

[string[]]$Servers,

# File with one server per line

[string]$ServersFromFile,

# Central Management Server

[string]$SqlCms

)

DynamicParam { if ($SqlCms) { return (Get-ParamSqlCmsGroups $SqlCms) } }

PROCESS {

if ([string]::IsNullOrEmpty($SqlCms) -and [string]::IsNullOrEmpty($ServersFromFile) -and [string]::IsNullOrEmpty($servers))

{ throw "You must specify a server list source using -Servers or -SqlCms or -ServersFromFile" }

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") -eq $null )

{ throw "Quitting: SMO Required. You can download it from <a href="http://goo.gl/R4yA6u">http://goo.gl/R4yA6u</a>" }

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Management.RegisteredServers") -eq $null )

{ throw "Quitting: SMO Required. You can download it from <a href="http://goo.gl/R4yA6u">http://goo.gl/R4yA6u</a>" }

$SqlCmsGroups = $psboundparameters.SqlCmsGroups

if ($SqlCms) { $servers = Get-SqlCmsRegServers -server $SqlCms -groups $SqlCmsGroups }

If ($ServersFromFile) { $servers = Get-Content $ServersFromFile }

$collection = @()

foreach ($servername in $servers) {

Write-Verbose "Attempting to connect to $servername"

$server = New-Object Microsoft.SqlServer.Management.Smo.Server $servername

try { $server.ConnectionContext.Connect() } catch { Write-Warning "Can't connect to $servername. Moving on."; continue }

$maxmem = $server.Configuration.MaxServerMemory.ConfigValue

$reserve = 1

$totalMemory = $server.PhysicalMemory

# Some servers underreport by 1MB.

if (($totalmemory % 1024) -ne 0) { $totalMemory = $totalMemory + 1 }

if ($totalMemory -ge 4096) {

$currentCount = $totalMemory

while ($currentCount/4096 -gt 0) {

if ($currentCount -gt 16384) {

$reserve += 1

$currentCount += -8192

} else {

$reserve += 1

$currentCount += -4096

}

}

}

$recommendedMax = [int]($totalMemory-($reserve*1024))

$object = New-Object PSObject -Property @{

Server = $server.name

TotalMB = $totalMemory

SqlMaxMB = $maxmem

RecommendedMB = $recommendedMax

}

$server.ConnectionContext.Disconnect()

$collection += $object

}

return ($collection | Sort-Object Server | Select Server, TotalMB, SqlMaxMB, RecommendedMB)

}

}

Function Set-SqlMaxMemory {

<#

.SYNOPSIS

Sets SQL Server max memory then displays information relating to SQL Server Max Memory configuration settings. Works on SQL Server 2000-2014.

.PARAMETER Servers

Allows you to specify a comma separated list of servers to query.

.PARAMETER ServersFromFile

Allows you to specify a list that's been populated by a list of servers to query. The format is as follows

server1

server2

server3

.PARAMETER SqlCms

Reports on a list of servers populated by the specified SQL Server Central Management Server.

.PARAMETER SqlCmsGroups

This is a parameter that appears when SqlCms has been specified. It is populated by Server Groups within the given Central Management Server.

.PARAMETER MaxMB

Specifies the max megabytes

.PARAMETER UseRecommended

Inspired by Jonathan Kehayias's post about SQL Server Max memory (<a href="http://bit.ly/sqlmemcalc">http://bit.ly/sqlmemcalc</a>), this uses a formula to determine the default optimum RAM to use, then sets the SQL max value to that number.

Jonathan notes that the formula used provides a *general recommendation* that doesn't account for everything that may be going on in your specific environment.

.NOTES

Author : Chrissy LeMaire

Requires:         PowerShell Version 3.0, SQL Server SMO, sysadmin access on SQL Servers

DateUpdated: 2015-May-21

.LINK

<a href="https://gallery.technet.microsoft.com/scriptcenter/Get-Set-SQL-Max-Memory-19147057">https://gallery.technet.microsoft.com/scriptcenter/Get-Set-SQL-Max-Memory-19147057</a>

.EXAMPLE

Set-SqlMaxMemory sqlserver 2048

Set max memory to 2048 MB on just one server, "sqlserver"

.EXAMPLE

Get-SqlMaxMemory -SqlCms sqlcluster | Where-Object { $_.SqlMaxMB -gt $_.TotalMB } | Set-SqlMaxMemory -UseRecommended

Find all servers in CMS that have Max SQL memory set to higher than the total memory of the server (think 2147483647),

then pipe those to Set-SqlMaxMemory and use the default recommendation

.EXAMPLE

Set-SqlMaxMemory -SqlCms sqlcluster -SqlCmsGroups Express -MaxMB 512 -Verbose

Specifically set memory to 512 MB for all servers within the "Express" server group on CMS "sqlcluster"

#>

[CmdletBinding()]

Param(

[parameter(Position=0)]

[string[]]$Servers,

[parameter(Position=1)]

[int]$MaxMB,

[string]$ServersFromFile,

[string]$SqlCms,

[switch]$UseRecommended,

[Parameter(ValueFromPipeline=$True)]

[object]$collection

)

DynamicParam { if ($SqlCms) { return (Get-ParamSqlCmsGroups $SqlCms)} }

PROCESS {

if ([string]::IsNullOrEmpty($SqlCms) -and [string]::IsNullOrEmpty($ServersFromFile) -and [string]::IsNullOrEmpty($servers) -and $collection -eq $null)

{ throw "You must specify a server list source using -Servers or -SqlCms or -ServersFromFile or you can pipe results from Get-SqlMaxMemory" }

if ($MaxMB -eq 0 -and $UseRecommended -eq $false -and $collection -eq $null) { throw "You must specify -MaxMB or -UseRecommended" }

if ($collection -eq $null) {

$SqlCmsGroups = $psboundparameters.SqlCmsGroups

if ($SqlCmsGroups -ne $null) {

$collection = Get-SqlMaxMemory -Servers $servers -SqlCms $SqlCms -ServersFromFile $ServersFromFile -SqlCmsGroups $SqlCmsGroups

} else { $collection = Get-SqlMaxMemory -Servers $servers -SqlCms $SqlCms -ServersFromFile $ServersFromFile }

}

$collection | Add-Member -NotePropertyName OldMaxValue -NotePropertyValue 0

foreach ($row in $collection) {

$server = New-Object Microsoft.SqlServer.Management.Smo.Server $row.server

try { $server.ConnectionContext.Connect() } catch { Write-Warning "Can't connect to $servername. Moving on."; continue }

if (!(Test-SqlSa $server)) {

Write-Warning "Not a sysadmin on $servername. Moving on."

$server.ConnectionContext.Disconnect()

continue

}

$row.OldMaxValue = $row.SqlMaxMB

try {

if ($UseRecommended) {

Write-Verbose "Changing $($row.server) SQL Server max from $($row.SqlMaxMB) to $($row.RecommendedMB) MB"

$server.Configuration.MaxServerMemory.ConfigValue = $row.RecommendedMB

$row.SqlMaxMB = $row.RecommendedMB

} else {

Write-Verbose "Changing $($row.server) SQL Server max from $($row.SqlMaxMB) to $MaxMB MB"

$server.Configuration.MaxServerMemory.ConfigValue = $MaxMB

$row.SqlMaxMB = $MaxMB

}

$server.Configuration.Alter()

} catch { Write-Warning "Could not modify Max Server Memory for $($row.server)" }

$server.ConnectionContext.Disconnect()

}

return $collection | Select Server, TotalMB, OldMaxValue, @{name="CurrentMaxValue";expression={$_.SqlMaxMB}}

}

}

Set-SqlMaxMemory $name 10000

Install Developer APPS

start-process F:\tools\7zipInstall.msi -ArgumentList "/q" -Wait

start-process F:\tools\ccleaner.exe -argumentlist "/S" -Wait

start-process F:\tools\fiddler4setup.exe -ArgumentList "/S" -Wait

start-process 'F:\tools\Firefox Setup Stub 36.0.4.exe' -ArgumentList "/S" -Wait

start-process F:\tools\iview438_setup.exe -ArgumentList "/silent" -Wait

start-process F:\tools\LINQPad4Setup.exe -ArgumentList "/silent" -Wait

start-process F:\tools\npp.6.7.5.Installer.exe -ArgumentList "/S" -Wait

start-process F:\tools\paint.net.4.0.5.install.exe -ArgumentList "/S" -Wait

start-process F:\tools\paint.net.4.0.5.install.exe -ArgumentList "/auto" -Wait

start-process F:\tools\PowerGUI.3.8.0.129.msi -ArgumentList "/q"

#start print spooler for cutepdf

net start spooler

sc query spooler

#fix hanging http://d4rkcell.com/archives/1217

start-process F:\tools\cutepdf-writer.exe -ArgumentList "/VERYSILENT /SUPPRESSMSGBOXES /NORESTART /SP-"

start-process F:\tools\VisualStudio2012\vs_premium.exe -ArgumentList "/adminfile AdminDeployment.xml /passive /norestart" -Wait -NoNewWindow -PassThru

$vsixInstallerPath = "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\VSIXInstaller.exe"

$extensionPath = "F:\tools\CKS.Dev11.vsix"

Start-Process -FilePath $vsixInstallerPath -ArgumentList "/q $extensionPath" -NoNewWindow -Wait

start-process F:\tools\SPDesigner\setup.exe -ArgumentList "/adminfile updates\adminfile.msp" -wait

start-process F:\tools\en_visual_studio_enterprise_2015_with_update_1_x86_x64_web_installer_8234346.exe -ArgumentList "/S /AdminFile F:\Tools\AdminDeployment.xml" -Wait

RDP to developer VM as svc_SPSetup and Launch AutoSPInstaller bat file

start-process F:\tools\AutoSPInstaller\SP\AutoSPInstaller\AutoSPInstallerLaunch.bat -wait

#Run autospinstaller

#Pre-reqs- 10 minutes w Restart

#Install binaries- 15 minutes

#Automation Fix- CU prompts for internet trusted file. 30 minutes (or right click all 3 CU files and go to Security and unblock, already done for June 2015 CU on AzureShare.).


#UPS Sync- we have to do this manually per install guide

#Add developer as full control of web applications

SharePoint- Configure User Profile AD Sync by hand

AD Connection:

Active Directory Company

Company.local

Company\svc_spups

passw0rd

(Sync All OUs)


Enable timerjob 1am daily:


Start full sync:


Manually add developer as full control of web applications.

Done!

Posted by: Eric Schrader | January 11, 2016

SharePoint DevOps Part 1 – Setup CentOS with Ansible

This is the first post in a series for creating a Linux host with Ansible to control Windows machines and install SharePoint.

Let me preface this with I have no clue how far I will get. I expect this to take a few weeks if it’s possible. Ansible seems to be growing, and this all might not be fully baked out. We will see! Feel free to comment.

  1. Part 1- Setup CentOS with Ansible (This post)
  2. Part 2- Setup Ansible with Windows Machines (Coming soon)
  3. Part 3- Using Ansible to prepare Windows Machines for SQL Server (Coming soon)
  4. Part 4- Using Ansible to install SharePoint using AutoSPInstaller (Coming soon)
  5. Part 5- Using Ansible to Maintain SharePoint Machines (Coming soon)

Introduction

There are various technologies I have attempted to use in the past for Windows Deployment SharePoint automation using AutoSPInstaller, such as:

  1. Chocolatey– I could not find enterprise worthy packages for SQL Server and SharePoint. I also couldn’t figure out a way to provision Azure VMs from it.
  2. PowerShell DSC – Way to overkill with push/pull when I just wanted to configure a new SharePoint farm
  3. Chef– Have not tried it. I personally can’t get over the names of things relating to items in the kitchen. But I believe this is a standard. Check it out
  4. Azure WebHooks– Runs PowerShell, possibly limited, on Azure VMs. Note each VM needs an endpoint for each runbook. I have 60 dev VMs/farms that I manage, and I wanted a different runbook for different parts of the SharePoint configuration process, this wasn’t going to happen.
  5. Azure RunBooks– Lots of great tutorials for building a SP Farm, but the farm was not fully configured. I prefer AutoSPInstaller and couldn’t figure out a way to merge them.

Now my above experience is very limited, as when I would hit a wall I would pretty much see how far the technology is and if it’s not supported I would give up. Also, the above experiences are a hybrid of on-premises SharePoint installation needs and Azure IaaS hosting SharePoint needs. So if this does not work, I will be going back to the drawing board or the above list. I have not used Microsoft System Center Orchestrator, as I rarely provision new Hyper-V VMs, and heard it is a ton of work to configure.

My Goal with Automation

I have a very well defined steps I have documented to install SharePoint 2013 farms with one or more servers, for dev/qa or production. These can be in Azure or on-premises in Hyper-V. I want something lightweight that I can provision a new VM, join it to the domain, create a domain if needed, run Windows Update, configure OS rules, install SQL Server (different versions), install SharePoint via AutoSPInstaller, patch SharePoint, and configure services. I would also like to set services and verify those rogue developers did not change any system settings (just kidding team). I know automation does not replace planning, but I hope to turn documentation and PowerShell scripts into a custom deployment tool I can use for provisioning future environments as well as maintaining existing environments. When I read the Phoenix Project, they brought up a good point of “If a family dog gets injured, you nurse it back to health, but if one of your cattle get injured, you will be having beef for dinner”, which basically means why are you spending 100 hours fixing an environment, when you can just recreate or replace it with another? Of course, developers and users change system settings, but the core of the machine can be recreated via a script in less than a day, rather than troubleshooting something and possibly not solving the issue.

Getting Started with Ansible

I picked up a book on Amazon called Ansible for DevOps by Jeff Geerling but shortly into the book realized it was only for Linux based machines. Ansible does support Windows targets, but the commands must be ran from a Linux OS. Dang. I don’t know Linux. So now I am writing this blog post. And it begins. Let’s see what Ansible can do for Windows targets.

Installing Linux – CentOS on a Windows Hyper-V host

  1. Download Cent OS7
    1. https://www.centos.org/download/
    2. I choose Torrent option, as the mirrors were pretty slow. Torrent went at 6mbps download and finished 4GB in about 10 minutes.
  2. Create new VM

    1. Choose a location for the VM file

    2. Choose Generation1

    3. Choose a fixed amount of ram

    4. Choose your Hyper V NIC

    5. Choose your VHD path/info for a new blank VHD to be created

    6. Choose the CentOS ISO file downloaded from the first step

    7. Turn on the VM!
    8. Boot to Cent OS install

Install CentOS on VM (then Python/Ansible)

  1. Select Language


  1. Choose software selection (Choose Server with GUI unless you know how to use Linux terminal well) I also selected the Development/Security tools, and MariaDB (I saw MariaDB in the Ansible book example and figure this will save me some steps later for Ansible testing)


  2. Choose disk


  3. Enable Ethernet and choose a hostname for the computer


  4. Verify everything looks good:


  5. While the OS is installing, configure a root and local user account. Root is like a local server admin password, which we will be using. The user account is a username and password, which you will be logging into each time you start the VM


  6. Select the blue Reboot option when the install is complete.

Log In to Cent OS

  1. Log in to CentOS
    1. License Agreement
      1. Hit 1 to read it, 2 to accept, c to continue, and c again to continue (I kinda struggled with this part)


    2. Sign in using the username and password you set up:


      1. Accept language, keyboard layout, and skip cloud accounts if desired. Then click Start using Linux!


  2. Run Terminal


Install Python/Ansible on CentOS

  1. Install Python
    1. Type SU then hit Enter in terminal to enter the root admin window


    2. Enter root password (different from user password, you entered it in setup)
    3. Once in root, install Python.
      1. Type: sudo yum install epel-release


      2. Hit Y to continue (twice)
      3. Verify complete

  2. Install Ansible
    1. Type sudo yum install ansible


    2. Select Y to continue (twice)

    3. Test ansible command to verify install is complete:
      1. Type: ansible –version


      2. You should get back a version number.

RESOURCES

Thanks to this article for the Python install help I was able to figure out how to install it without errors on CentOS7. Here are the same commands over again, just together without screenshots:

http://stackoverflow.com/questions/32048021/yum-what-is-the-message-no-package-ansible-available

$ su

$ sudo yum install epel-release

$ sudo yum install ansible

If you are a SharePoint admin or use, you have probably seen this error message on your SharePoint 2013 MySite Newsfeed:

“We’re still collecting the latest news. You may see more if you try again a little later.”

sharepoint 2013 mysite company newsfeed error -Were still collecting the latest news

Hopefully this article will help explain a few common scenarios I have ran into, and how to resolve any errors. Please post if you have any tips or suggestions, as I am always looking for thoughts from the community on this.

This article will address:

  • Common reasons for newsfeed data not displaying
  • What is distributed cache?
  • Distributed cache configuration
  • Shutdown/Reboot WFE procedure for maintenance so you don’t lose your cache
  • Repopulating cache (if server stopped unexpectedly)
  • References

Common reasons for newsfeed data not displaying

  1. Someone rebooted all your Distributed Cache servers at the same time
    1. Check the task manager uptime to see how long the servers with distributed cache have been running, or if they were rebooted unexpectedly
    2. Fix: Repopulate cache using PowerShell, or maybe wait a long time for new news
    3. Prevent it from happening again: Shut down one server at a time, stopping the cache first, rebooting, and then starting the cache again.
  2. “Everyone” is empty because it only keeps company conversations for 14 days by default.
    1. Fix: Increase the retention time, or encourage people to post to the company newsfeed (not site newsfeeds). See this article for more information on what appears in site newsfeeds vs company newsfeeds. https://support.office.com/en-za/article/What-items-appear-in-your-newsfeed-bd3d9268-0408-4ad4-bc51-2e4ec5406e16#__toc327280723
  3. Distributed cache is not configured right
    1. Fix: Configure it right J this one is so simple, yet so difficult I find. See configuration below.

What is distributed cache?

Distributed cache is a framework Microsoft uses to quickly host social information in SharePoint within the SP servers ram. This can be enabled on one or many SP servers in your farm.

Official definition can be found on this poster, https://www.microsoft.com/en-us/download/details.aspx?id=35557

What uses distributed cache?

Pretty much anything social, but some of the social data comes from content databases and user profile databases. Company newsfeed posts are stored in distributed cache.

  • Newsfeeds
  • Authentication
  • OneNote client access
  • Security Trimming
  • Page load performance

Distributed cache configuration

Note: Run any scripts/commands logged in as the SPFarm account, and be sure to run SharePoint Management Shell as administrator if you have UAC enabled (like a good administrator)

Caution: configuration deletes the cache, so you will need to repopulate the cache after configuring it.

Determine servers to host the distributed cache service

Usually the WFE servers, not servers running search or excel. AutoSPInstaller has a limit of 4 servers, but typically it does not configure distributed cache correctly.

Configuring Distributed Cache

There is a good article here on these commands, https://technet.microsoft.com/en-us/library/JJ219613.aspx and probably better than this article I am writing. But it’s very long so I wrote this article to get Admins 90%-100% of the way there.

Here is how I have been configuring distributed cache. Thanks Jon for the help!

  1. Use Central Admin or PowerShell to start/stop the SharePoint Distributed Cache service on the desired servers in your farm (usually WFE’s).
    1. Or you can use PowerShell to get/start-spserviceinstance of Distributed Cache on the desired servers. I like to use PowerShell to see what servers are running Distributed Cache within my farm:
      1. Get-SPServiceInstance | where-object {$_.typename -ilike “*distributed*”}
  2. Verify Cache service is running on desired servers:
    1. Use-CacheCluster
      1. Note, this command doesn’t configure the server, but just connects the current PowerShell session to manage the cache cluster it’s joined for the PowerShell management session. It’s actually an alias command for Connect-AFCacheClusterConfiguration.
    2. Get-CacheHost
    3. You should see each server running distributed cache listed above. If not, there might be more work to configure the cache cluster I may have missed in this post. Let me know!
  3. Get current configuration
    1. Get-AFCacheHostConfiguration -ComputerName wfe01 -CachePort “22233”
    2. The Cache Size can be updated, see guide here https://technet.microsoft.com/en-us/library/JJ219613.aspx#memory . For 16GB of ram on our WFE servers, we go with 819MB (~5% of 16GB). Note, changing this requires the distributed cache service to be stopped on the computer you are changing it on. Update-SPDistributedCacheSize -CacheSizeInMB 819
  4. Export config, verify service account for distributed cache, as well as servers.
    1. Export-CacheClusterConfig -Path C:\test.xml
      1. Check max cache size (default is 5% -, no more than 4GB – size depends on services on the server)
      2. Check servers – ensure only WFE (or desired servers are in the cluster)
      3. Check service account – ensure all servers use the same service account (spservice)
      4. Check ports
  5. Warning: After configuration is complete do not ever run Add-SPDistributedCacheServiceInstance or Remove-SPDistributedCacheServiceInstance. It reconfigures the cluster (and usually incorrectly)

Shutdown/Reboot WFE procedure

If you have to do reboots on the WFE for Windows Updates, etc., you might be expecting to lose your Newsfeed cache. Here is the proper procedure to retain the cache.

Summary: shutdown the cache one server at a time, reboot that server, add the server back into the cache cluster. Repeat on next server.

  1. Verify Cache service is running on desired servers (more than one server too is key):
    1. Use-CacheCluster
    2. Get-CacheHost
  2. Reboot SQL Server first if needed. Get this out of the way.
    1. Wait for SQL Server to come back online
  3. Reboot WFE1
    1. Perform these commands on WFE1
    2. Verify Cache service is running on desired servers (more than one server too is key):
      1. Use-CacheCluster
      2. Get-CacheHost
    3. Run Stop-spdistributedcacheserviceinstance -graceful
    4. Verify Cache service is stopped on WFE1. Ensure it is stopped before proceeding:
      1. Get-CacheHost
    5. Reboot WFE1
    6. Verify Cache service is running on WFE1:
      1. Use-CacheCluster
      2. Get-CacheHost
      3. If not, go to Central Admin Services on Server and start Distributed Cache service on WFE1, or use PowerShell.
  4. Reboot WFE2
    1. (Repeat above Step #3, but replace WFE1 with WFE2)
  5. Verify it is running
    1. Verify Cache service is running on desired servers (more than one server too is key):
      1. Use-CacheCluster
      2. Get-CacheHost

Repopulating cache (if server stopped unexpectedly)

Replace URL with your mysite URL. This script will populate each user’s cache using Update-SPRepopulateMicroblogFeedCache and the entire user profile newsfeed cache using Update-SPRepopulateMicroblogLMTCache. I am not sure if I stole this script from anywhere, but part of it is from various user profile scripts adapted to fix the users feed cache.

Note: Run any scripts/commands logged in as the SPFarm account, and be sure to run SharePoint Management Shell as administrator if you have UAC enabled (like a good administrator). Otherwise you will get a .ctor error that will drive you crazy.

Download the script from here: http://pastebin.com/K9yR2pEk

$proxy
= Get-SPServiceApplicationProxy | ? {$_.Name -ilike
“User Profile Service Application*”}

Update-SPRepopulateMicroblogLMTCache -ProfileServiceApplicationProxy $proxy

[System.Reflection.Assembly]::LoadWithPartialName(“Microsoft.Office.Server”)

[System.Reflection.Assembly]::LoadWithPartialName(“Microsoft.Office.Server.UserProfiles”)

$url
=
http://mysiteurl.domain.com

$contextWeb
=
New-Object
Microsoft.SharePoint.SPSite($url);

$ServerContext
= [Microsoft.Office.Server.ServerContext]::GetContext($contextWeb);

$UserProfileManager
=
New-Object
Microsoft.Office.Server.UserProfiles.UserProfileManager($ServerContext);

$Profiles
=
$UserProfileManager.GetEnumerator();

foreach ($oUser
in
$Profiles ) {


if ($oUser.item(“SPS-PersonalSiteCapabilities”).Value -eq 14 ){


$personalurl
=
$url
+
$oUser.item(“personalspace”).Value


Write-Host
$oUser.item(“AccountName”).Value

Update-SPRepopulateMicroblogFeedCache -ProfileServiceApplicationProxy $proxy -accountname $oUser.item(“AccountName”).Value


#-siteurl $personalurl

}

}

$contextWeb.Dispose()

After running the script on each WFE where distributed cache runs, wait 15 minutes for the Newsfeed data to populate.

Then test newsfeed.

References

http://consulting.risualblogs.com/blog/2014/04/01/export-impor-distributed-cache-configuration-in-sharepoint-2013/

http://sharepoint.stackexchange.com/questions/125798/userprofileapplicationnotavailableexception-logging-userprofileapplicationpro

http://netwovenblogs.com/2014/03/11/the-newsfeed-is-not-working-on-mysite-in-sharepoint-2013/

Older Posts »

Categories