• ThinIO facts and figures, Part 1: VDI and Ram caching.
• ThinIO facts and figures, Part 2: The Bootstorm chestnut.
• ThinIO facts and figures, Part 3: RDS and Ram caching.

In the final blog post in this series, we’re going to discuss storage design and a frequent problem face when sizing storage. Lets get right into it:

“Designing for average, is designing for failure”

Peak IOPS:1015, Average IOPS: 78

A frequent mistake I see customers and consultants alike make is taking an average of a sizing requirement and using that as a baseline for sizing environments.

Looking at the figures produced from our internal load tests, we saw just an average of 78 IOPS required on write from a vanilla server without ThinIO to provide storage from this XenApp server.

Now frequently, people will take a figure like that, throw in 25% for growth and bob’s your uncle, ‘order the hardware Mr SAN man’. When I have questioned them about that assumption, they’ll often respond “oh it will be a bit slow if theres contention but it’ll even itself out”.

Right? Wrong.

Things don’t go slow when they are over subscribed, they stop.

Don’t take my word for it! Lets do some simple theoretical math:

If you take a storage device and allocate 100 IOPS to this machine, what’s going to happen when a peak like 1000 IOPS is requested? A lot of queuing. 

In theory, keeping to the 100 IOPS figure, that 1 second burst IO is now taking over 10 seconds to satisfy (1000 / 10).

But it gets worse, all subsequent IO that is requested after that spike occurred is going to also be haulted waiting for this task to occur.

Assuming you’re now mid spike and 10 seconds later the request is finished… taking your average figure, you now have 10 seconds worth of 100 IO’s per second potentially queued up behind…

Low and behold another login occurs and? STOP. Storage timeouts, twirly whirlies, application crashes, hour glasses and the good old “logins are slow”.

Oh ok, So how do I size to accommodate this? 

Well you’re between a rock and a hard place aren’t you. You can’t tell users when to login, the price tag of a SAN sized for peak activity + 20% is going to cost you more than your entire desktop estate. And as you can see, it’s never safe to assume it will run slow. 

Buying into shared storage is a tricky business 

Storage is expensive. Very expensive. It always annoys me when you hear vendors in this space refering to themselves as ‘reassuringly expensive’. To me this directly translates to ‘We can charge what we want, so we will and you can be reassured we feel the price tag is worth it.’

Storage was never written with desktop workloads in mind, it was written for ‘steady state’ server workloads and was in the progress of going the way of the ‘dodo’ (extinct) up until that first release of vMotion requiring shared storage, which some say was the saving of the market.

Many vendors are going with software or hardware intelligent tiering. This is a great feature, but the real question to ask is how frequently data is moved from the hot tier to lower tier? Press your vendor on this as they more than likely wont know! Microsoft storage spaces is a prime example of this with a really poor optimisation process of just once a day!

Then ask yourself what happens when a base image update occurs and changes the disk layout of the base golden image? Further, stateless technologies from the bigger vendors delete the differencing disk on restart, can you be sure the new disk is going to end up in the smaller, faster SSD or RAM tier? Or is data up there already in contention?

RAM is far less tricky

RAM is commodity, available in abundance and throughout every virtual desktop project I’ve architected and deployed, you run out of CPU resources in a ‘fully loaded’ host way before you will run out of RAM. RAM has no running maintenance cost, Ram is an upfront CAPEX cost and requires little to no maintenance.

The beauty of what ThinIO does with the little resources you assign it, is turn that desktop workload into a healthier and happier server workload of minimal burst IO and a low steady state IO requirement.

Note the peak of just 40.5 IOPS and average IOPS of less than 2.

with as little as just 200mb cache for each of the 10 users logging in, within an aggressive 3 minute window, we reduced the peak from 1000 to 40. That’s a 96% reduction in burst IO.

With ThinIO, you:

• reduce your exposure to massive IO spikes.
• Improve user logon times.
• significantly reduce your daily IOPS run rate.
• Increase user productivity by spending less time waiting for the storage.
• Commit to nothing up front, test it and see how well it works. If you are happy, then buy in.

Lots of Intelligence baked in:

ThinIO is acutely aware of key operating system events that will cause these kind of spikes and react accordingly to reduce the spikes in IOPS created. ThinIO constantly watches the behavior and IO pattern of the storage and tunes itself accordingly.

Unlike other technologies, ThinIO is a true caching and performance solution. We do not move useful data in and out of the cache on demand when cache availability is contencious. We track patterns and frequency of block access to respond accordingly, delivering all the benefits we have mentioned, even with the tiniest cache, without EVER reducing the capability of the storage when overwhelmed.

And on the opposite side of the scale, when underworked, we leverage our cache to deliver deeper read savings as above.

ThinIO also has a powerful API and PowerShell interface to allow you to report and interact with the cache on demand.

Wrap up:

And with the end of the series looming, allow me to finish on some easy points:

 ThinIO Allows you to:

• size your SAN outside of the Lamborghini category & price tag for your desktop estate.
• rapidly achieve far deeper density on your current hardware when you are feeling the Pinch.
• guarantee a level of performance by assigning cache per VM, disallowing other users to steal or hamper caching resources.
• Improve user experience and login times immediately.
• Reduce the impact of boot storms and similar IO storm scenarios.

No other vendor can offer as quick a turn around time with their product. ThinIO installs in seconds and offers a huge range of compatibility.

One more thing:

In case you missed ThinIO’s launch day at E2EVC Barcelona, ThinIO is now in GA, available from our website and production ready! More marketing to follow! But grab your copy now and get playing!

To recap, below you will find the previous articles:

• ThinIO facts and figures, Part 1: VDI and Ram caching.
• ThinIO facts and figures, Part 2: The Bootstorm chestnut.

Off topic note:

two years ago at an E2EVC event, the concept behind ThinIO was born with just a mad scientist idea amongst peers.

If you are lucky enough to be attending E2EVC this weekend, David and I will be there presenting ThinIO and maybe, just maybe there will be an announcement. Our session is on Saturday at 15:30 so pop by, you won’t be disappointed.

Back on topic:

So here’s a really interesting blog post. Remote Desktop Services (XenApp / XenDesktop hosted shared) or whatever you like to call it. RDS really presents a fun caching platform for us, as it allows us to deal with a much higher IO volume and achieve deeper savings.

We’ve really tested the heck out of this platform for how we perform on Microsoft RDS, Horizon View RDS integration and Citrix XenSplitPersonality with Machine Creation Services.

The figures we are sharing today are based on the following configuration and load test:

• Citrix XenDesktop 7.6
• Windows Server 2012 r2
• Citrix User Profile Manager.
• 16gb of Ram.
• 4 vCpu.
• LoginVSI 4.1 medium workload 1 hour test.
• 10 users.
• VMFS 5 volume.

Fun figures!

Diving straight in, lets start by looking at the volume of savings across three cache types.

Reviewing the details for a moment:

Running repetitive tests of at least 3 per cache type, we found even at the lowest entry point we would support (50mb per user) we saw phenomenal savings of over 70% on write IO.

No pressure no diamonds!

To put that into perspective, at a 512 MB cache for 10 users, our cache reached maximum capacity at the second user login. With 8 users still left to login, cache full and still an hours worth of load testing left, our ThinIO technology was under serious pressure.

This is key to why ThinIO is such a great solution. We won’t just perform great until we fill our cache, we don’t require architecture changes or care about your storage type, we have no lead times or install days, we will carry on to work with what is available to use, to take a large ammount of pressure off storage IOPS and data throughput.

With the figures above, you can see just how well the intelligence behind our cache can scale even when it faces such a steep workload.

Below you will find a breakdown of each test:

512 MB cache:

Breaking down into the figures, on the 512mb cache test, it’s clear to see just how well ThinIO deals with the tiniest of caches:

When we side by side this with our baseline averages, you can see we take a huge chunk out of that Spiky login pattern and continue to  reduce the steady state IO as the test continues:

So lets move up and see how we get on!

1024 mb cache:

Doubling up our cache size we see a great increase in both read and write savings as you’d expect.

With 100mb of cache per user, and the average user profile in the test 3 times that size. We are still under pressure. As we will natively favour optimisations to write IO over read, you’ll see the bulk of improvements happen in write when we’re under pressure as illustrated in this test:

With more cache available during the peak IO point, we make further savings on write:

2048 mb cache:

and at our recommended value of 200mb per user in Remote Desktop Services, the results are phenomenal! With this size, even still below the 300mb mark per user profile, the read IO gets a really good boost and the write IO saving well over the 95% mark!

And the side by side comparison is every bit as good as the savings illustrated above, reducing that peak bursty IO to just 41 IOPS:

But there’s more! 

As i pointed out in the previous blog, IOPS are just one side of the story. A reduction of data throughput to the disk is also a big benefit when it comes to storage optimisation, and as you can see we make a big difference:

Wrap up:

So there you have it, with ThinIO, a simple, in VM solution, you can you seriously reduce your IO footprint, boost user performance and achieve greater storage density per virtual machine or on Remote Desktop Services technology.

In the mean time:

If you would like a chance to test ThinIO pre-release, find access to the public beta below. Thank you for your time and happy testing!

A link to the updated article is here. Best of luck!

A

Not happy with the Irk, and still on my app developing buzz, i decided to write SBC Printers:

SBC-Printers is a simple little .net 4 application, leveraging WMI for printer enumeration and control.Because SBC Printers is an executable, it can published as a XenApp application. Sbc Printers can also be installed as the default printers interface on the start menu:

So really your users won’t know the difference or care for that matter!

SBC-Printers also comes with securable options for adding or deleting local printers:

The display of add or delete can be controlled via the settings file in the installation directory:

Installation:

1. Download the following MSI
2. Install the MSI to the default directory.

To restrict the standard printers dialog from users, but leaving it accessible to administrators:

• Browse to c:program files (x86)SBC-Printersbin

• run the powershell script below, make sure to run it as an administrator!

That’s it, once the Powershell script runs. it removes the users access to the registry classes giving them access to the standard devices and printers interface. Which means we’re now ready to provision SBC-Printers to replace it.

Provisioning the replacement to the user:

Now just import the userkey.reg into the users profile on login, you can do this via your user profile manager of choice, or use Group Policy preferences.

That’s it!

As you can see I haven’t streamlined the install process too much, this is mostly down to the simplicity of the tool. If you like SBC-Printers but would like a better installer, just drop me a comment below.

Roll back:

if you need to restore the standard interface, uninstall SBC-Printers then add the (local computerusers) group back to the following registry keys ACL:

•  HKCRsoftwareclassesCLSID{A8A91A66-3A7D-4424-8D24-04E180695C7A}
• HKCRsoftwareWow6432NodeCLSID{A8A91A66-3A7D-4424-8D24-04E180695C7A}

Having been in this situation recently, I needed to audit and report on the types of files open on the file server, my hunch was a certain select number of users were running applications (like *gulp* lotus notes) from the network share.

Disappointed with the powershell scripts on the interwebs, I decided to write my own function to perform this task:

[sourcecode language=”powershell”]
function get-openfiles{
param(
$computername=@($env:computername),
$verbose=$false)
$collection = @()
foreach ($computer in $computername){
$netfile = [ADSI]"WinNT://$computer/LanmanServer"

$netfile.Invoke("Resources") | foreach {
try{
$collection += New-Object PsObject -Property @{
Id = $_.GetType().InvokeMember("Name", ‘GetProperty’, $null, $_, $null)
itemPath = $_.GetType().InvokeMember("Path", ‘GetProperty’, $null, $_, $null)
UserName = $_.GetType().InvokeMember("User", ‘GetProperty’, $null, $_, $null)
LockCount = $_.GetType().InvokeMember("LockCount", ‘GetProperty’, $null, $_, $null)
Server = $computer
}
}
catch{
if ($verbose){write-warning $error[0]}
}
}
}
Return $collection
}
[/sourcecode]

The function above (get-openfiles) has been written to accept an array of servers to the command line and it will return the following items:

• The ID of the open file.

• The server it’s open from.

• The username who has the file open.

• The amount of locks the file has.

A couple of quick examples for using this command are below:


Retrieving open files from server1:






[sourcecode language=”powershell”]get-openfiles -computername server1 | select server,itempath,lockcount[/sourcecode]



Retrieve a count of open files that end with the nsf file type (Lotus Notes):






[sourcecode language=”powershell”](get-open files -computername server1,server2 | ? {$_.itempath -like "*.nsf*"}).count()[/sourcecode]



Retrieve a report of total open files on a number of file servers:



[sourcecode language=”powershell”]get-openfiles -computername server1,server2,server3,server4,server5 | group -property server[/sourcecode]

With great pleasure I’m announcing the general availability of ThinKiosk 3.1. Quite a bit of change under the hood and some nice features added to match.

New features:

VMware View enhanced support:

VMware View has gotten some love in this update, A big thanks to Jarian Gibson for the help.

You can now enforce end of session options for VMware view:

You can also now choose to wipe the last users details from the Kiosk between View sessions:

FTP policy management:

With ThinKiosk 3.1, you no longer are tied to manage the thinkiosk devices by Group Policy or local registry settings, you can now also use an ftp server with a shared xml configuration file:

Just configure a Device as you would like it to appear, unlock the admin menu and you can export the configuration to xml:

Then move it to your ftp server!

Encryption:

The unlock password in group policy can now be encrypted to save it appearing in plain text to anyone capable of viewing the policy. ThinKiosk 3.1 ships with a password encryption tool you can use to encrypt your password.

You can also test reversing the password to plain text to make sure you get it right before applying it en-mass and locking yourself out!

This encryption functionality has now been added to both the offline configuration tool:

And by default the FTP password will be encrypted too!

Battery Awareness:

ThinKiosk is now aware of batteries in laptop devices and will report their status.

When the battery begins to run out, ThinKiosk will throw a warning in the foreground as below:

You can additionally disable this functionality with the offline configuration tool.

Pre launch Citrix Receiver:

A rare issue seen with the latest versions of the receiver was a bit of a hang, pause or complete lock up as receiver came to life. To combat this, you can now choose to early launch the receiver for Citrix, allowing it to gracefully start up in the background before the user requires it.

Early launch process:

A number of customers needed to have third party software launched as soon as ThinKiosk started each day. I’ve now added the ability to early launch a process 

You can also choose to launch this process as hidden, away from the user.

Browser navigation buttons:

ThinKiosk can now act as a locked down browser by adding back and forward buttons.

AM / PM clock:

This feature was asked for quite a few times, so now you can set the clock to 12 hour.

Debug Mode:

A fully fledged debug window has been added to help timing issues. The debug menu can be accessed via command line (-debug) or via the admin menu in ThinKiosk.

Zorder awareness:

In rare situations (and I’ve been unable to reproduce it) ThinKiosk can jump above the citrix session when a log off of the web interface happens or during the login process.

Zorder awareness will tell ThinKiosk to send itself to the back of the Zorder when the browser finishes rendering. It will also display a hide button, which will send ThinKiosk to  the back in this rare event.

Please use this setting as a troubleshooting tool, not a production setting. If this setting fixes the issue for you, please drop me an email and I’ll write it in. As I’ve been unable to reproduce this issue, it’s a bit rough around the edges.

Citrix Storefront timeout screen:

ThinKiosk is now aware of the timeout screen and will automagically redirect back to the login screen if it see’s it.

Hide ThinKiosk when a desktop is active:

If you wish to outright hide ThinKiosk while a desktop is active, you can now do so!

Even More sites:

Support for up to 20 sites has been added, thanks Martijn!

Sticky Home Page:

A request came through to allow the home page always be site 1, this has now been included.

Bug Fixes:

• support for environment variables in custom tools and prelaunch commands. (thanks Nathan).
• Offline config tool not setting password correctly.
• VB Powerpack accidentally bundled with ThinKiosk 3.0
• In process launch mode, power options were intermittently being applied.

And it’s still free!

ThinKiosk development has taken quite some time and it takes time to support you via email. If you use ThinKiosk in your environment or appreciate the savings its made for you, please consider making a donation or paying for enterprise support to help me keep this project alive… I would really appreciate it as it will allow me to invest in better development tools to make the product look and feel even better!

Hello there, and thank you for dropping back for part 3…

I suppose I should start with the disappointing news that I have yet to test this option for VDI in a box. And despite Aaron Parker’s suggestions it wasn’t due to lack of inspiration, it was down to lack of time! This series has gathered allot of interest from both community and storage vendors alike, and I feel I should set the record straight before I got any further:

1. This isn’t a production idea, you would be crazy to use this idea in a live environment.
2. Throughout this entire project, we’re focusing on pooled stateless. Stateful desktops would be a separate post entirely.
3. This wasn’t an attack on products in this market space, merely a fresh view on an old problem.
4. If i had the skills or funds necessary to run this project to a production solution, I wouldn’t have posted it. I would already be hard at work creating a reasonably priced product!

Now that my declarations are out of the way, I’d first like to talk about the moral of the story. This isn’t an unfamiliar expression:

IOPS mitigation is not about read IOPS it’s about WRITE IOPS!

VMware, Citrix and Microsoft have similar but very different solutions for read IOPS negotiation. Similar in the sense that they try to negate storage read IOPS. But the key difference with XenServer is the local disk cache via Intellicache has the out of box functionality to cache the majority of read’s to local disk (think SSD*) without the baked in soft limit of 512 MB for Microsoft HyperV and VMware respectively.

Long story short, VMware and Microsoft’s solution is about 512mb of recognizable read IOPS negation un-tuned, but enabled. Of course this value can be tuned upwards, but the low entry point of cache would suggest, at least to me, that tuning up will have an upsetting affect on the host.

This to me is why IntelliCache has the upperhand in the (value add product) VDI space for read IOPS negation and they even throw in the Hypervisor as part of your XenDesktop licensing, so win win, but what about those pesky write IOPS?

Let’s look at Citrix for a moment!

If we review article 1 and article 2 for a moment, the half baked idea formed by Barry Schiffer, Ingmar Verheij, Kees Baggerman, Remko Weijnen and I, when combined with Citrix IntelliCache turned out to be a phenomenal combination to create an IOPS avoidance product. But the key point we had in the back of our heads was, Citrix provisioning server already has this driver!

Citrix provisioning server has a RAM cache driver, with configurable size baked into the Provisioning Services product. This driver works in a very similar way to the EWF driver, just without the API and flexibility of a Microsoft driver.

There are customers of Citrix out there using RAM caching with XenApp who I spoke to, they have assigned a large (8gb+ cache, 3-4gb of cache utilised day to day) to negotiate the chance of a spillover, but it could still happen and this is an acceptable risk to their implementation. I struggled to find a XenDesktop customer using this method who would talk about their implementation.

But with XenDesktop in mind, using an overly large cache size to avoid spillover just doesn’t scale the way a consolidated Server Based Computing model would… Which leaves us back in this same dilemma, I’d like to cache in RAM, but I don’t want to risk users losing data when they perform a task we have not factored for.

So with all this in mind, lets talk about my final hurdle I faced with this project.

RAM cache flood and my half baked solution:

The Microsoft EWF driver suffered the same problem as the Citrix provisioning server when the cache filled, i.e. it froze or outright bug checked if you tried to reference a file put into the cache before you filled it with other information.

Jeff Silverman has done a great article on what happens in this event with the Citrix Provisioning Services RAM cache and I urge you to go read it so I don’t have to repeat it! Go now, I’ll wait right here for you to get back!

Read that article? Ok good, let’s continue.

To combat this scenario with the Windows EWF driver, I created a very rough around the edges windows service using polling (Sorry Remko) to check the size of the cache periodically via the EWFAPI and write the cache out to disk.

The function in the EWFAPI called EwfCommitAndDisableLive allowed me to dump the ram cache to disk and subvert the cache, going straight to disk when this event had occurred. By using this routine the ram cache simply disables itself and allows pass-through to disk at this point.

With this in mind, I tuned my service to spill to disk when the cache was greater than 80% of ram available, not in use by the operating system when the system booted, This worked well to a point, but the key failure to this approach became apparent when you opened a large number of applications and they struggled to occupy the space available around the cache.

My second attempt however, proved much more fruitful where I monitored for free system bytes in memory and if this dropped below a certain threshold the EWF drive began it’s dump routine to disk. Once the dump routine was complete, ram cleared and the writes had been committed to storage where the disk continued to use it for the remainder of the session. Once this spill over had occurred, a notification bubble fired in the user session warning them of the degradation of service and they should log off saving work at the next convenient moment…

Voila! no blue screen, spill to disk and user notification of degradation.

This wasn’t fool proof, it was very rough, it didn’t handle files being written larger than the RAM cache, but In my opinion, it satisfied the biggest fear and business case against using the Citrix Provisioning server ram caching, the ram cache flood scenario. I was happy with the results and it scaled well to the size of my lab, it showed what is possible with a RAM filter driver and it allowed me to prove my point before I started to poke the hornets nest of the storage market. So let’s park the EWF story for now and focus on my favorite subject, Citrix technologies.

Note: I won’t be making this service and driver solution publicly available, it’s not a production ready solution and Microsoft would sue the pants off me. I’m sure you’ll understand why, but if you want more information drop me an email.

The next part’s of this blog are all theoretical, but I know certain people I want to listen, are listening (Hi Kenneth, hope you are well :)).

Negating the negated. Lets talk about that spill over.

But what about that Spill over event? by using a spill over from ram to disk, we create a situation where we could change a steady, slightly reliable IOPS per from each desktop, to a situation where, “couldn’t a collection of spillovers at the same time cause your storage to becoming swamped with IOPS?”

Absolutely, and I’ve been thinking about this quite a bit… But there’s another hidden potential here even in a RAM spill over scenario…

With a little bit of trickery couldn’t we also negate this spillover event with a simple provisioning job?

With a bit of time spent thinking about this scenario, this is what I came up with…

Why doesn’t the controller, for XenApp or XenDesktop (PVS / DDC) copy or (my personal preference) create a local, blank differencing disk when a VM boots?

The hypervisor could be completely agnostic at this point, we could spill over to local disk, keeping write IOPS completely away from the shared storage? even in a spill over event?

This approach (in this half baked enthusiasts view) would negate the negated… But don’t take my word for it, lets go for some theoretical solutions.

Solution A: Implementing a spill over routine with the Citrix provisioning server driver.

So with my success with the Microsoft driver, I got myself thinking how easy would this be to do utilizing the Citrix Provisioning Services driver? Without access to the code, I’m going to be making slightly risky statements here, but I have allot of faith in Citrix that they could make this happen.

From everyone I have spoken to about this idea, they all see the value in the ability to spill out of RAM… So Citrix, please make it so. Below are some idea’s for deployment methods assuming Citrix do march down this route and the pro’s and con’s I see that live with each scenario.

Bear in mind, I’m not a storage guy, a full time developer or a software architect, I’m just an enthusiast that see’s potential, so drop your comments below as to what you think!

Machine Creation Services.

Idea A: Provisioning services improved RAM caching, Machine Creation services and Intellicache on XenServer.

Utilizing XenServer and MCS we could leverage the intellicache to negate the read IOPS as we proved in my own EWF driver testing, but still deliver on that spill over mechanism allowing continuation of service.

Pros:

• Read IOPS negation.
• RAM caching to remove write IOPS (bar a spill over).
• Continuation of service in a cache flood scenario.

Cons:

• Limited to XenServer.
• Over provisioning of RAM necessary per desktop.
• RAM spillover will result in a large amount of IOPS to the shared storage.

Idea B: Provisioning services improved RAM caching, Machine Creation services and Intellicache on XenServer… With local copy!

Same benefits as previous, but now, we have zero reliance on the shared storage when the VM is up (except for ID disk actions).

Pros:

• Read IOPS negation.
• RAM caching to remove write IOPS.
• Uses local resources in a spill over.
• Continuation of service in a cache flood scenario.

Cons:

• Limited to XenServer.
• Over provisioning of RAM necessary per desktop.

So that’s MCS in a nutshell with per VM caching, I think this solution has so much potential I can’t believe it’s not been done, but I digress. So lets park that topic for now and move on to Citrix Provisioning Services.

Citrix Provisioning Services:

So lets look at the “favorite of many” technology.

Idea A: Citrix Provisioning Services and Improved RAM caching driver.

In a pure Provisioning services environment, we would force our read IOPS via the lan, instead of storage protocol but still deliver a spill back to disk to allow continuation of service.

Pros:

• Hypervisor agnostic.
• RAM caching to remove write IOPS (bar a spill over).
• Continuation of service in a cache flood scenario.
• Potentially no shared storage needed, at all, if caching on the PVS server.

Cons:

• Read IOPS aren’t really negated, they’re just forced over another technology.
• Over provisioning of RAM necessary per desktop.
• RAM spillover will result in a large amount of IOPS to the shared storage / pvs server.

Idea B: Citrix Provisioning Services and Improved RAM caching driver.. With local copy!

Taking the above benefits, but with the gain of utilizing local storage in the spillover event.

Pros:

• Hypervisor agnostic.
• RAM caching to remove write IOPS.
• Uses local resources in a spill over.
• Continuation of service in a cache flood scenario.
• Potentially no shared storage needed, at all.

Cons:

• Read IOPS aren’t really negated, they’re just forced over another technology.
• Over provisioning of RAM necessary per desktop.

So lets Review:

And there we have it, 4 solutions to IOPS negation utilizing the Provisioning server RAM caching driver, with a little bit of a modification to deliver a robust solution to RAM caching.

The copy and creation of differencing disks again would deliver additional benefits to leverage the hardware you put into each Hypervisor without the Shared Storage investment.

Win Win, but is it?

There’s an oversight here:

There’s a niggle here that’s been bothering me for some time and you’ll probably note I mentioned it as a CON to most of the solutions above… I’m going to lay it out on the table in complete honesty…

“Isn’t over provisioning RAM on a per desktop basis a waste of resource? Wouldn’t it be better if we could share that resource across all VM’s on a Hypervisor basis?”

You betcha! If we are assigning out (for argument sake) 1gb of ram cache per VM, that RAM is locked into that VM and if another machine spills, the free RAM in other desktops is wasted.

You would be completely insane not to reserve this RAM per VM, if an over-commit for the VM’s is reached this RAM will merely spill out to a page file type arrangement, negating all your benefits.

Ultimately, assigning RAM in this way could be seen as wasteful in the grand scheme of things…

But there are other ways to skin this cat!

So this leads me on to something I was also considering which popped up in a twitter conversation recently. RAM disk’s and Hypervisors.

Current storage vendors will create a storage pool, consisting of ram inside a VM, per hosting hypervisor for local storage to allow good IOPS leveraging that VM. This VM can perform additional RAM optimizations like compression, de-duplication and sharing of similar pages to reduce the count.

This approach is super clever, but, In my humble opinion (please don’t kill me vendors), this approach is wasteful. Running a VM as a storage repository per host has an overhead to run this virtual machine and takes away from the agnostic solution that is possible…

What If the hypervisor provides the RAM disk?

So excluding ESXi for a second, as getting a RAM disk into that platform would require VMware to allow access to the stack. Lets look at Hyper-V (2012) and XenServer for a second…

With Unix and Windows platforms, RAM disks have been available for years. They were once a necessity and a number of vendors still provide them for high performance IO environments.

Lets say (for arguments sake) Citrix and Microsoft decide to provide a snap-in to their hypervisor to allow native RAM disks (or a vendor writes one themselves!) and maybe, they even decide to provide RAM compression, Spill over to local disk, dedupe, and page sharing on this volume from the hypervisor stack…

Wouldn’t this extend provide all the benefits we’ve spoken about, without the need for a VM per host? And using Thin Provisioning allow all desktops to share the large pool of RAMdisk available?

Yes, yes, it would.

Above is an example of how I would see this working.

So random pictures are fine and all, but what about the read IOPS negation technologies? and what about combining these with XenServer or Hyper-V?

XenServer and IntelliCache:

Well there you have it now, all the benefits of a per VM filter, leveraging intellicache for reads and spilling out to local disk… RESULT!

Pro’s:

• Read IOPS negation
• Write IOPS negation
• No Shared Storage required for running VM’s
• Shared pool of RAM for all desktop’s to use.

Con’s:

• A small one, but no migration of VM’s.

Provisioning server?

and again, all the benefits of a per VM filter, reads redirected via lan and spilling out to local disk… RESULT!

Pro’s:

• Write IOPS negation
• No Shared Storage required for running VM’s
• Shared pool of RAM for all desktop’s to use.

Con’s:

• A small one, but no migration of VM’s.
• No real read IOPS negation.

And HyperV + CSV Cache?

Well here’s an accumulation of my thoughts:

So lets just talk for a second about what we are seeing… Utilizing a RAM disk with spill over, and copied vhd’s on boot we are removing the need for shared storage completely and hosting the IOPS from the hypervisor, natively without the need for additional VM’s.

And see that little icon down the bottom? Yep, that’s right, live migration from host to host thanks to Microsofts Shared Nothing Live Migration!

Pro’s:

• Some write IOPS negation
• No Shared Storage required for running VM’s
• Liv migration.
• Shared pool of RAM for all desktop’s to use.
• write back to local disk.

Con’s:

• No full read IOPS negation.

Review.

I’m sure there’s loads of reading in here, and there will be tons of thought and questions after this blog post. This has been in my head for roughly 9 months now and it feels victorious to finally get it all down on paper.

At the end of the day, Ram Caching is the way of the future, you pay no maintenance on it, you keep your IOPS off your storage and with a little magic from Microsoft, or particularly Citrix. You could see these benefits.

If / When these technologies make it out to you, you could quite happily stick with Machine Creation services, leverage intellicache and your shared storage requirement could be a cheap NAS from any vendor. Provisioning services also see’s allot of benefits from this approach, but the real creme de la creme is in that intellicache feature.

The key functionality to these approaches is simple:

• Cache RAM
• Spill to disk
• Keep the differences on local storage if it spills.

One last thing?

In true Mark Templeton fashion, you know I have a one last thing. This actually hit me today while writing this post and to be honest, I’m so amazed by the potential of this idea I’m going to build it in the lab this weekend.

But until then, a teaser…

What if I told you there was already an overlay, freely available baked into a Microsoft technology that would allow you to overlay local storage in Hyper-V with RAM. A powerful API attached to it, and the ability to write specific files to the local disk, subverting the overlay only when needed?

• RAM disk type technology? > yep
• already available? > yep
• Powerful API? > yep
• already can spill to disk? > yep

Yep Yep Yep! check back in a few days.

(If you know what it is already, keep it to yourself until I’ve gotten out of the lab!)

What do you think?

It doesn’t take a half baked enthusiast like me to see this potential and I’d be really eager to hear your comments on these approaches. If you would prefer to keep your comments offline, you can reach me on andrew@andrewmorgan.ie.

Before I launch into my little discovery, here’s something I want to share:

Since ThinKiosk was released in January, It’s been downloaded over 5,000 times and I have counted (only from those who have contacted me) that there are well over 10,000 instances running in customer environments to this day. Version 1 was released with just 700 lines of code and version 2.3 is just shy of 6,000. This is absolutely amazing to me, seeing an idea that I thought was a “Publish and Forget” blog post be embraced so passionately by the community. So for this, I just wanted to thank you guys for all your help, support and idea’s.

Anyway, back to it. While reviewing my emails recently, it struck me that there are many, many customers using ThinKiosk, not on old PC’s which I had written the program for, but on Thin Clients from top vendors… Now this doesn’t bother me one bit, as the more people using this tool the better, but why aren’t these people using Linux Thin Clients? I’m a large advocate of linux based thin clients in my day job, hell, they’re much cheaper, easier to manage and in most cases boot faster… Why choose Windows?

So confused and curious, I decided to perform a little poll on Twitter:

"Monday morning poll, why do you choose Windows based thin clients?"

And to my surprise, the feedback was great, below I’ve included the top reasons why community members choose Windows Thin Clients:

• HDX Redirection (Aero, Flash, Printing, Scanning)
• Better feature set on Windows / Future proof for upcoming features.
• Central Management via Active Directory / Group Policy.
• Driver support (proxy card’s, smart card’s)
• Familiar User Interface.
• Familiar support platform / Unified support platform / No in house linux knowledge.

So all this got me thinking, even with a list that long of why Windows based thin clients are preferred, why are these guys using ThinKiosk on out of box Thin Clients? Surely a paid for solution will be an end to end solution?

Well not really and why is this? Citrix receiver.

Citrix receiver for Windows (Previously the Program Neighbourhood agent) has been designed for published applications running inside of the users local desktop session, not for Thin Clients connecting to virtual desktops and this is very clear when you consider that Receiver will by default place your published desktop on the start menu or desktop of your session.

This approach will normally lead you to have to log the user into the Thin Client as themselves, which you would prefer to be locked down in the first place. This also leaves you with the challenge of how do you log them off after their desktop session has ended!

Sure Citrix have added some additional desktop related functionality along the way (Desktop Viewer) but even desktop viewer itself is designed for running inside a users session allowing the user to jump back to the local device via the home button.. which can’t subsequently be locked down sadly.

Citrix did also release the desktop lock tool, which is good for very small use cases, but lacks the functionality of multiple desktops, workspace control, user customisations etc… Hence why ThinKiosk came to be!

Thin Client vendor work around?

Most Thin Client vendors will allow you to present glorified shortcuts to ICA files on the desktop of the Thin Client device, or auto launch them on boot… But this approach eliminates the benefits of Workspace Control, XenApp preferencial load balancing and requires trickery to get pass through authentication to work… Not only this but managing these shortcuts in a multi desktop and multi language environment where users roam from country to country is a complete administrative nightmare!

But What about the web access products from Citrix?

Now the obvious alternative to the Citrix Receiver is the Citrix’s web access platforms… The web interface or Cloud Gateway, unlike the desktop lock, or ica files offers multiple desktops, workspace control, load balancing policies etc. You can also leverage web interfaces built in password changing feature for the user with them having to be logged in to the local device and even allow them to reset their own password or unlock their account with Citrix Single Sign on!

And the best part is? the users will already be very familiar with this interface if you have an access gateway or Secure gateway for remote access.

Aha! now it makes sense!

I accidentally provided an easy to use, unified access approach across all windows devices…and I feel blind for not seeing it before!

What ThinKiosk also accidentally addressed, was allowing this web access platform to be leveraged with ease, security and minimal configuration… from any windows platform, Thin Client or old pc.

So in short, I think this was the success story for ThinKiosk I hadn’t considered… so much so that I’ve changed my own approach and mindset for Linux based Thin Clients too, locking down a local copy of  Firefox and presenting the Web Interface or Cloud Gateway.

So if you’re considering rolling in windows Thin Clients for your current or next VDI project… Consider using ThinKiosk, it’ll save you alot of pain, will work seamlessly with all your clients (thin or fat), and will save you time in management in the long run!

Throughout this blog post, I’m going to be talking about Process Affinity and Process Priority. Understanding these definitions will help. I’m also only targetting server 2008 R2 and Windows 7 for this post.

An issue I see time and time again in Virtual Desktop Infrastructure and Server Based Computing environments is CPU spikes cause sessions to pause and stick. These CPU spikes can be caused by overcommitment or overzealous application usage and the vendors (Citrix, ThreadMaster, RES, Appsense, etc) have come to the rescue frequently with tools to reduce these events from occurring.

The problem with these solutions (Appsense excluded) is they are reactive, I.E. they take a number of seconds to identify a spike and respond by dropping the priority of the process or reducing the Users time on the processor.

These few seconds of high usage, reduce the amount of time the users applications and desktop session can spend on the processor, this reduction of resources cause the VDI desktop of the user to be completely unusable during the pause and ultimately resulting in complaining from the user community. Worse yet, if this is an SBC environment you now have all users on that server locked out!

And all this may sound a bit over the top, but its very easy to reproduce. For example, create a Microsoft Excel sheet and attempt to vlookup over a few thousand cells and watch the processor get chewed up. In this example, yes a vLookup across that many cells would be better served from a server based solution, but what’s stopping your users doing it?… Exactly.

Looking at SBC for a second:

With Server 2008 R2, the Fair Share CPU scheduling feature really is best (free) solution out there for CPU Utilisation management. The processor scheduling is far better than the Citrix equivalent and they also offer a feature (albeit, it feels unfinished) called Windows System Resource Manager to configure the processor affinity along with Process soft rules for performance management.

By setting these extremely intensive Multi-threaded applications to an affinity, we can “Lock” these processes to only run on certain processor cores, allowing the remaining cores to rebalance the rest of the workload. This ultimately, leaves the users session still responsive even if the application no longer responds. Allowing the user to continue working in other application until the intensive process is complete.

The problem with Windows System Resource Manager?

• It’s a pain to configure on more than one system
• It requires (yeuck) Windows Internal Database
• It has been marked as “Depreciated” in Server 2012
• If a user changes affinity, Resource manager doesn’t re adjust.
• It still suffers from “pauses” or sluggishness when the Cpu is really being hammered.

This was my dillema recently with both XenApp & Xendesktop. Ultimately not happy to Pay for a solution, embrace a depreciated tool, or wash my hands of the problem, I decided to write my own tool, complimenting fair share CPU scheduling but allowing affinity and priority locking.

Introducing Threadlocker:

ThreadLocker has been written to help you target known problematic processes and deal with them pro actively.

Threadlocker also has the added benefit of dropping the priority of known grunty processes to idle, meaning the process will use as much CPU as it can, but any other higher priority process can interupt without sluggishness or pauses.

With ThreadLocker you can target these processes and set their Affinity number of processors they can run on:

Or their Process Priority to drop them out of the normal running context:

• Threadlocker is light weight and scalable.
• Threadlocker works flawlessly with Cpu Fair Share Scheduling, so even if a process is “ThreadLocked” the users running those applications will fair share on the designated cores.
• Threadlockers configuration is xml based and can be copied down to the machine with Startup script or Group Policy preferences.
• Threadlocker can be used in VDI environments where no other free solution is available.
• Threadlocker will re-adjust priority, or affinity if the devious user tries to remove the restriction.

Threadlocker has been successfully tested on the following platforms:

• XenApp 6.5
• Windows Server 2008 R2 Service Pack 1
• Windows 7 x64 Service Pack 1

• Threadlocker Requires .Net Framework 3.5 sp1

Download:

Here

Version 2.0 is a complete rework of the code as I adopted some standards, a big thank you again to Pierre Marmignon for taking the time to point me in the right direction!

ThinKiosk 2.0 is stuffed with new features and supported platforms, here’s a quick list:

• Enterprise Software Support options.
• New Supported VDI solutions.
• Power Management.
• Offline Configuration tool.
• Custom Title.
• Site selection list.
• Custom tools menu.
• Home Button.
• Local Printer management.
• Desktop Mode (log off the web interface when a session starts).
• Auto log off redirect.
• Best practice group policies settings.
• Group Policy to lock down everything!

For full information on the changes, see the About and Features pages.

To grab a copy and start playing, go here: (note, recreate the group policies in a new OU)

A big thank you to my translators and Beta testers, there was just too many of you to mention!