Inconvenient MVC Web Optimization Framework and High CPU Utilization

While I am writing this post I am bit relaxed and listening to 'bollywood' music so don't wonder if you find anything off topic or typos :), lazy weekend!!

What was the issue?

Last week, while working with the Window azure cloud services we tried to stress test it and guess what we found? CPU spikes!!
Initially we thought that this might be something related to load on the server as it was a small VM instance but at the same time the results were unbelievable as number of concurrent requests to the server were comparatively less. This gave us the kick start and we started to drill down to the real issue.

The approach:

It all started with several meetings with few brainy people around as I was clueless, some even directly said man your application got memory licks..but then
Thanks to the Visual Studio profiling framework which helped me a-lot to drill down into real issue about this CPU spikes. You can read about how to use this feature and how it helps in my blog post here Profiling windows azure cloud services - http://passionatetechie.blogspot.in/2014/01/profiling-for-windows-azure-cloud.html

When I tried to stress test the service and tried to collect the profiled logs , found that there were these two methods which were taking almost more than 80% of CPU, wowwwwww!!
1. Scripts.Render
2. Styles.Render

 Note - I was using ASP.NET MVC web optimization nuget version 1.0

Irony is, these methods are from the web optimization framework of ASP.NET MVC which is suppose to improve the performance of your web application by bundling the java-scripts and css files. More info about bundling here - http://www.asp.net/mvc/tutorials/mvc-4/bundling-and-minification

All right then of course the straight way to remove these calls from application and try to stress application again and check results. we did exact same thing and this time application performed pretty well!! no CPU spikes!! man bingooo!!!!!!

But then why in my application the optimization framework was doing something like this? why ? why why and why.......? this question even appeared in my dreams!!
I started search around it but didn't find something helpful, I even started doubting my decision about removing those calls from the application unless I hit this post here
http://stackoverflow.com/questions/14210721/mvc-4-on-azure-scripts-render-bundles-jquery-slow and
http://stackoverflow.com/questions/12230246/azure-cache-preview-outputcache-high-cpu-slow

Here is why and it is purely based on what I understood

When you use the bundling (Web optimization framework) - server uses the caching.
But now as you are mentioning the setting in your web.config file that you are not using default caching of server anymore instead you want to use the Windows azure distributed cache.
Like this

<caching>
  <outputCache defaultProvider="DistributedCache">
    <providers>
      <add name="DistributedCache" 

type="Microsoft.Web.DistributedCache.DistributedCacheOutputCacheProvider, 

Microsoft.Web.DistributedCache" cacheName="default" dataCacheClientName="default" />
    </providers>
  </outputCache>
</caching>

So seems this web optimization framework doesn't understand this distributed cache and it simply ignores the caching part.
Due to this it tries to recompile your bundles every time with each request which causes higher CPU utilization.


I also had the distributed cache related settings in my application's web.config , so I decided to remove both from my application i.e. Bundling as well as distributed caching - it worked and CPU utilization never touched the boundary.


Takeaways:

In case you are facing the same issue of high CPU consumption on the web server in your ASP.NET MVC application or cloud service then
1. Run the VS Profiler locally and see whats the root cause of it
2. If you find the Optimization framework APIs are taking most of the CPU then try removing it and stress your application again to see the results.

Next step:

According to this link http://aspnetoptimization.codeplex.com/workitem/46 , its the discussion on codeplex on very same issue but If you go to the comments section then someone said that they have fixed it in the next version of nuget. (i.e. post version 1.0)
I havent tried using next version of this nuget but when I will do it I will update this post.

Hope this helps and stops wondering someone about why my CPU percentage is higher when using web optimization framework of MVC.

Profiling Windows Azure Cloud Services

Hi again guys,

Last week I was working really close with performance related things of the web applications and this what I came across for Windows Azure Cloud services.

Profiling is one of the great feature of Visual Studio and I almost fell in love with it. why? because first of all its quite descriptive and helps to measure the memory / CPU monitoring of your application and second is it can be configured in both ways i.e. while running your application locally as well as while running your application in the cloud.

I will not go into much deeper on how to use this feature and how it helps, all you need to do is click on the link to know and you are done.
http://msdn.microsoft.com/en-us/library/2s0xxa1d.aspx

You can use this great feature to go into the quite deeper understanding in terms of how your application is consuming resources on the server and it helps in deciding to scale up or scale down your application needs. More information on it at http://msdn.microsoft.com/en-us/library/dd264994.aspx

But all right , lets come back to the Cloud arena and let see how can use this for Windows Azure cloud services.

basically you can use this feature while deploying your application using VS to cloud deployment wizard. (Publish)

A great article http://msdn.microsoft.com/en-us/library/windowsazure/hh369930.aspx#BK_ProfilingCloudService which explains the approach about how you can do it pretty easily while deploying to Azure through your VS.

I was interested more on what special this VS does when we check the check box of Enable Profiling in the publish wizard? and here is what I found

Basically VS adds few entries in your cloud service definition and cloud service configuration files.

(I am using Windows Azure SDK 2.2)

In csdef
 <InternalEndpoint name="Microsoft.WindowsAzure.Plugins.Caching.cacheArbitrationPort" protocol="tcp" />
  <InternalEndpoint name="Microsoft.WindowsAzure.Plugins.Caching.cacheClusterPort" protocol="tcp" />
  <InternalEndpoint name="Microsoft.WindowsAzure.Plugins.Caching.cacheReplicationPort" protocol="tcp" />
  <InternalEndpoint name="Microsoft.WindowsAzure.Plugins.Caching.cacheServicePortInternal" protocol="tcp" />
  <InternalEndpoint name="Microsoft.WindowsAzure.Plugins.Caching.cacheSocketPort" protocol="tcp" />

In cscfg
<Setting name="Profiling.ProfilingConnectionString" value="YourStorageAccount" />
<Setting name="CloudToolsDiagnosticAgentVersion" value="2.2" />

But then I also found that the size of the deployment packages increases when you enable this option of profiling, and that's because VS also adds the 64 bit profiler in your package.
So I am not sure about how you can use this feature when you dont want to publish using VS to your cloud service, however article http://michaelwasham.com/2011/08/10/deployment-to-windows-azure-fails-with-profiling-enabled/ explains how can you do it but as it is written for older version of SDK so I am doubtful if this works for later versions of Windows Azure SDK.

Overall profiling feature managed to impress me with with it's native integration with Windows Azure Cloud Services.

Once you enable the profiling and deploy the cloud service, VS 2012 Server explorer helps you in getting report out of your profiling build with just a single click and you are set to analyze the report.

Converting Lists to Collections in C# and Using Linq on it

All right guys, this one is one of the shortest blog post I am doing today and hope it helps people like me who wasted almost few minutes searching for solution on this.

Basically .NET Framework offers us too many collections however two of those are
1. List
2. Collection

We all know Linq, i.e. this nice and slick feature of .NET to query various collections however when it comes to Collections then we get this famous typecasting error
i..e Cannot convert IEnumerable to Collection etc etc..

Well, after several minutes of discussion with my brainy peers and some search on internet here is what I have done

Its a workaround which works in two steps
1. Convert your Collection to List and perform Linq on it.
2. Convert back your List to Collection. How? here we go

Collection<Entity> collection = new ObservableCollection<Entity>(collection.ToList().Distinct());

And you are done.

Hope this helps someone.

Working with boolean values and hidden fields in MVC 4

Hi guys,

This is quite simple but I learned something new in MVC today, so thought to share with you all.

I was trying to assign a Boolean field to a hidden field using ViewBag in MVC

<input type="hidden" id="somehiddenfield" value="@ViewBag.MyBooleanValue" />

And now when I was trying to read the value using jQuery it was always returning me hidden field’s value as “value” which was quite strange to me as I was expecting true or false.

var hiddenVal = $("#somehiddenfield").val();

After a quick search, came to know that this has been changed a little in MVC 4 , now it follows the checkbox behavior.

i.e. you will get the hidden field’s value as “value” when the Boolean value will be true and nothing when Boolean value will be false.

So you might need to change the implementation a little if you explicitly want to read the true and false as hidden field values from jQuery.

<input type="hidden" id="somehiddenfield " value="@(ViewBag.MyBooleanValue.ToString())" />

More details at - http://stackoverflow.com/questions/13455270/why-is-my-hidden-input-writing-value-value-instead-of-true-false

 

Hope this helps someone.

Unable to open TFS Build Process Templates in Visual Studio 2012

Hi guys,

Yet another development issue and solution

Issue:

After installing Visual Studio 2012 , I was not able to open the build process templates on my environment. Strange this was - they were getting opened correctly in other environments or on other machines.

What was the error - 

System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.SharePoint.WorkflowExtensions, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.VisualStudio.SharePoint.WorkflowExtensions, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'

What was the issue and solution?

Well, not quite sure about the exact issue but only solution which I could find to resolve this error is to install the workflow manager on your environment.
Here is how you can do it

Launch you command prompt (as administrator) and run this command

C:\Program Files\Microsoft\Web Platform Installer>WebpiCmd.exe /Install /Products:WorkflowManagerTools /AcceptEula

After doing this I was able to open build process templates on my environment.
Hope this helps you.

Reference - Here

Error Solved : http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name was not present on the provided ClaimsIdentity.

While working with MVC application , I came across an interesting thing and got something to learn from it so thought to share.

Scenario :

Typically when you implement any MVC web application , you want to implement some security features in it and hence use of anti-forgery token is one of the approach I was trying to implement in one of my MVC web application.

How it works?

Internally how it works is , in traditional web application which are not claims aware – it simply uses User.Identity.Name as anti-forgery token to validate form submitted.  

But when we try the same with claims aware applications– it throws an error. 

Why? 

Because now it tries to use the claims of type NameIdentifier and IdentityProvider (by default).

Error:

‘http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name' was not present on the provided ClaimsIdentity.

Solution:

Either your claims provider should send you the claims of type NameIdentifier as well as IdentityProvider , but in my case I was not having both claims with me.

So I had to use the following workaround to resolve this issue -

Add the following line in the App_Start method of the application.

AntiForgeryConfig.UniqueClaimTypeIdentifier = "http://your-sts.net/user/EmailAddress";

As the name suggest - it makes application aware that the unique claim type provider is EmailAddress and not the default one.

After this change , you can see the __RequestVerificationToken on the details page source information.

Conclusion:

This Error can be solved by letting application know that which is the claim type you want to use as unique identifier. In my case I am using EmailAddress because I had this type of claim available with me so you can also use any other claim type which your sts is providing you.

TFS Build Error Solved: Set-AzureDeployment : Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host

Hi guys,

Recently I faced a strange issue and figured out a solution as well so thought to share it because I didn't find much over the web about it.

We have created the TFS Build System built for on premise TFS server. We are using it for continuous deployment on Windows Azure Cloud Services and everything is working fine. There is an excellent documentation available here on MSDN so you can take a look at it.

What was the error?
The issue came in when build stopped working , prior builds were successful but we started facing issue with each new build.

Error Message was - Set-AzureDeployment : Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host.

Actually in the background - build process has a workflow and it tries to execute an PowerShell command on build server for deployment of your cloud services in Windows Azure.

What is the solution?
The way build process workflow works is - it needs the .publishsetting file for doing deployments on your Azure Cloud Services.
All you need to do is , download the .publishsetting file for your Azure subscription (you can click here) and save it locally. Now go to the build server and replace your old .publishsetting file with this new one.
Now try queuing your builds - everything should work fine (If It would have worked before :) ).

If you are not using the build system but only using the PowerShell command i.e. Set-AzureDeployment to do the deployment then also you can opt the same solution , i.e. just download and replace your old .publishsetting file with new one.

Another solution
Now , even though the solution mentioned above worked for me fine - but I faced similar issue on another environment few days after I written this post and above solution didn't work so had to search for solution again and found a workaround.
So thought to edit the blog post and document this workaround so that it helps someone.

The changes needs to be done in the script - what this changes is - try to upload the cspkg file to blob storage and the create / updnate the deployment using blob url.

You can define the new variable $containerName at top of your file and initialize with some string value which will act as container name.

# If the target container does not already exist, create it. 
$containerState = Get-AzureStorageContainer -Name $containerName -ea 0
if ($containerState -eq $null)
{
    New-AzureStorageContainer -Name $containerName | out-null
}

Set-AzureStorageBlobContent -File $packageLocation -Container $containerName -Blob $blob -Force| Out-Null

$script:packageLocation = (Get-AzureStorageBlob -blob $blobName -Container $containerName).ICloudBlob.uri.AbsoluteUri

Hope this helps someone.