• ThinIO facts and figures, Part 1: VDI and Ram caching.
• ThinIO facts and figures, Part 2: The Bootstorm chestnut.
• ThinIO facts and figures, Part 3: RDS and Ram caching.

In the final blog post in this series, we’re going to discuss storage design and a frequent problem face when sizing storage. Lets get right into it:

“Designing for average, is designing for failure”

Peak IOPS:1015, Average IOPS: 78

A frequent mistake I see customers and consultants alike make is taking an average of a sizing requirement and using that as a baseline for sizing environments.

Looking at the figures produced from our internal load tests, we saw just an average of 78 IOPS required on write from a vanilla server without ThinIO to provide storage from this XenApp server.

Now frequently, people will take a figure like that, throw in 25% for growth and bob’s your uncle, ‘order the hardware Mr SAN man’. When I have questioned them about that assumption, they’ll often respond “oh it will be a bit slow if theres contention but it’ll even itself out”.

Right? Wrong.

Things don’t go slow when they are over subscribed, they stop.

Don’t take my word for it! Lets do some simple theoretical math:

If you take a storage device and allocate 100 IOPS to this machine, what’s going to happen when a peak like 1000 IOPS is requested? A lot of queuing. 

In theory, keeping to the 100 IOPS figure, that 1 second burst IO is now taking over 10 seconds to satisfy (1000 / 10).

But it gets worse, all subsequent IO that is requested after that spike occurred is going to also be haulted waiting for this task to occur.

Assuming you’re now mid spike and 10 seconds later the request is finished… taking your average figure, you now have 10 seconds worth of 100 IO’s per second potentially queued up behind…

Low and behold another login occurs and? STOP. Storage timeouts, twirly whirlies, application crashes, hour glasses and the good old “logins are slow”.

Oh ok, So how do I size to accommodate this? 

Well you’re between a rock and a hard place aren’t you. You can’t tell users when to login, the price tag of a SAN sized for peak activity + 20% is going to cost you more than your entire desktop estate. And as you can see, it’s never safe to assume it will run slow. 

Buying into shared storage is a tricky business 

Storage is expensive. Very expensive. It always annoys me when you hear vendors in this space refering to themselves as ‘reassuringly expensive’. To me this directly translates to ‘We can charge what we want, so we will and you can be reassured we feel the price tag is worth it.’

Storage was never written with desktop workloads in mind, it was written for ‘steady state’ server workloads and was in the progress of going the way of the ‘dodo’ (extinct) up until that first release of vMotion requiring shared storage, which some say was the saving of the market.

Many vendors are going with software or hardware intelligent tiering. This is a great feature, but the real question to ask is how frequently data is moved from the hot tier to lower tier? Press your vendor on this as they more than likely wont know! Microsoft storage spaces is a prime example of this with a really poor optimisation process of just once a day!

Then ask yourself what happens when a base image update occurs and changes the disk layout of the base golden image? Further, stateless technologies from the bigger vendors delete the differencing disk on restart, can you be sure the new disk is going to end up in the smaller, faster SSD or RAM tier? Or is data up there already in contention?

RAM is far less tricky

RAM is commodity, available in abundance and throughout every virtual desktop project I’ve architected and deployed, you run out of CPU resources in a ‘fully loaded’ host way before you will run out of RAM. RAM has no running maintenance cost, Ram is an upfront CAPEX cost and requires little to no maintenance.

The beauty of what ThinIO does with the little resources you assign it, is turn that desktop workload into a healthier and happier server workload of minimal burst IO and a low steady state IO requirement.

Note the peak of just 40.5 IOPS and average IOPS of less than 2.

with as little as just 200mb cache for each of the 10 users logging in, within an aggressive 3 minute window, we reduced the peak from 1000 to 40. That’s a 96% reduction in burst IO.

With ThinIO, you:

• reduce your exposure to massive IO spikes.
• Improve user logon times.
• significantly reduce your daily IOPS run rate.
• Increase user productivity by spending less time waiting for the storage.
• Commit to nothing up front, test it and see how well it works. If you are happy, then buy in.

Lots of Intelligence baked in:

ThinIO is acutely aware of key operating system events that will cause these kind of spikes and react accordingly to reduce the spikes in IOPS created. ThinIO constantly watches the behavior and IO pattern of the storage and tunes itself accordingly.

Unlike other technologies, ThinIO is a true caching and performance solution. We do not move useful data in and out of the cache on demand when cache availability is contencious. We track patterns and frequency of block access to respond accordingly, delivering all the benefits we have mentioned, even with the tiniest cache, without EVER reducing the capability of the storage when overwhelmed.

And on the opposite side of the scale, when underworked, we leverage our cache to deliver deeper read savings as above.

ThinIO also has a powerful API and PowerShell interface to allow you to report and interact with the cache on demand.

Wrap up:

And with the end of the series looming, allow me to finish on some easy points:

 ThinIO Allows you to:

• size your SAN outside of the Lamborghini category & price tag for your desktop estate.
• rapidly achieve far deeper density on your current hardware when you are feeling the Pinch.
• guarantee a level of performance by assigning cache per VM, disallowing other users to steal or hamper caching resources.
• Improve user experience and login times immediately.
• Reduce the impact of boot storms and similar IO storm scenarios.

No other vendor can offer as quick a turn around time with their product. ThinIO installs in seconds and offers a huge range of compatibility.

One more thing:

In case you missed ThinIO’s launch day at E2EVC Barcelona, ThinIO is now in GA, available from our website and production ready! More marketing to follow! But grab your copy now and get playing!

To recap, below you will find the previous articles:

• ThinIO facts and figures, Part 1: VDI and Ram caching.
• ThinIO facts and figures, Part 2: The Bootstorm chestnut.

Off topic note:

two years ago at an E2EVC event, the concept behind ThinIO was born with just a mad scientist idea amongst peers.

If you are lucky enough to be attending E2EVC this weekend, David and I will be there presenting ThinIO and maybe, just maybe there will be an announcement. Our session is on Saturday at 15:30 so pop by, you won’t be disappointed.

Back on topic:

So here’s a really interesting blog post. Remote Desktop Services (XenApp / XenDesktop hosted shared) or whatever you like to call it. RDS really presents a fun caching platform for us, as it allows us to deal with a much higher IO volume and achieve deeper savings.

We’ve really tested the heck out of this platform for how we perform on Microsoft RDS, Horizon View RDS integration and Citrix XenSplitPersonality with Machine Creation Services.

The figures we are sharing today are based on the following configuration and load test:

• Citrix XenDesktop 7.6
• Windows Server 2012 r2
• Citrix User Profile Manager.
• 16gb of Ram.
• 4 vCpu.
• LoginVSI 4.1 medium workload 1 hour test.
• 10 users.
• VMFS 5 volume.

Fun figures!

Diving straight in, lets start by looking at the volume of savings across three cache types.

Reviewing the details for a moment:

Running repetitive tests of at least 3 per cache type, we found even at the lowest entry point we would support (50mb per user) we saw phenomenal savings of over 70% on write IO.

No pressure no diamonds!

To put that into perspective, at a 512 MB cache for 10 users, our cache reached maximum capacity at the second user login. With 8 users still left to login, cache full and still an hours worth of load testing left, our ThinIO technology was under serious pressure.

This is key to why ThinIO is such a great solution. We won’t just perform great until we fill our cache, we don’t require architecture changes or care about your storage type, we have no lead times or install days, we will carry on to work with what is available to use, to take a large ammount of pressure off storage IOPS and data throughput.

With the figures above, you can see just how well the intelligence behind our cache can scale even when it faces such a steep workload.

Below you will find a breakdown of each test:

512 MB cache:

Breaking down into the figures, on the 512mb cache test, it’s clear to see just how well ThinIO deals with the tiniest of caches:

When we side by side this with our baseline averages, you can see we take a huge chunk out of that Spiky login pattern and continue to  reduce the steady state IO as the test continues:

So lets move up and see how we get on!

1024 mb cache:

Doubling up our cache size we see a great increase in both read and write savings as you’d expect.

With 100mb of cache per user, and the average user profile in the test 3 times that size. We are still under pressure. As we will natively favour optimisations to write IO over read, you’ll see the bulk of improvements happen in write when we’re under pressure as illustrated in this test:

With more cache available during the peak IO point, we make further savings on write:

2048 mb cache:

and at our recommended value of 200mb per user in Remote Desktop Services, the results are phenomenal! With this size, even still below the 300mb mark per user profile, the read IO gets a really good boost and the write IO saving well over the 95% mark!

And the side by side comparison is every bit as good as the savings illustrated above, reducing that peak bursty IO to just 41 IOPS:

But there’s more! 

As i pointed out in the previous blog, IOPS are just one side of the story. A reduction of data throughput to the disk is also a big benefit when it comes to storage optimisation, and as you can see we make a big difference:

Wrap up:

So there you have it, with ThinIO, a simple, in VM solution, you can you seriously reduce your IO footprint, boost user performance and achieve greater storage density per virtual machine or on Remote Desktop Services technology.

In the mean time:

If you would like a chance to test ThinIO pre-release, find access to the public beta below. Thank you for your time and happy testing!

• ThinIO facts and figures, Part 1: VDI and Ram caching.

Getting right to it:

In this industry when somebody says ‘boot storms!’ – most of us will respond with:

Boot storms are a well documented, boring problem and have many solutions available from vendors and hypervisors alike. Most solutions today rely on a ‘shared memory’ storage area to cache ‘on boot’, in theory caching only one startup or one pattern in order to then serve it back to the proceeding desktops to boot.

But why are boot storms an issue? While working on ThinIO we had the unique ability to really dive into the Windows boot process and analyse why boot storms cause the damage they do and in this post we thought we’d share our findings to better document the issue.

Boot data:

Taking a typical windows 7 boot, to the login screen and idling until all services have started, the data traversing from disk to VM is relatively small. in our testing we found an average of Just 500-600 mb of data is read during this process, and write data barely registers at between 20 and 30mb.

But hey, what gives? Taking such low data throughput, why is boot such a contenscious issue? Have I been misled with marketing and vendor nonsense?

The IO chestnut:

Sadly no, it’s the way windows requests this data, but don’t take my word for it…. Behold, the incredible mess that is the Windows boot process!

Yep, that’s right, in the time Windows requested roughly 600mb of data, it sent down an astounding 70 thousand IO’s in the space of 2-3 minutes!

Math time:

Now if you were to take these figures as they stand, you would take 70,000 IO’s divide this into 560mb and you’d probably end up with an average of about 8k of data requested per IO… You’d be wrong.

As my good buddy Conor Scolard would say, ‘when you Assume, you make an ass out of you and me’.

To better understand the bounderies of Windows, Windows requests IO’s between the minimum of 512 bytes all the way up the spectrum later in the boot process to 128k and above. But it requests these blocks sparcely, on demand, and not just once per sector, the same blocks are frequently accessed.

The net result of this causes absolute havok on the storage:

The crux of the issue is, for each one of these IO’s, the storage provider needs to compute the block data requested, seek the data out, then return it.

But 70,000 of these IO operations for a meagre 600mb of data is madness and you can now see exactly why boot storms were labelled as such for those early adopters who had their hands burned by this fact finding mission.

I’ll mitigate this issue by just booting my VM’s at night!

I’m sure you will! I would also love to see your face if a number of users happen to restart their desktops during the day, cascading 70,000 IO’s per desktop to the storage in a 2 minute window, per desktop!

Bootstorming IS an issue.

Now, knowing all this, it makes sense as to why storage and hypervisors alike are using a cache of ram.

But how does ThinIO fit in here? With Read Ahead of course!

Knowing the Windows boot process as intimate as only a technology like ThinIO can, there are many, many optimisations we can make to this process.

We can both speed the boot process up and also massively reduce the storage requirement while in VM, without any fancy caching mechanism!

With ThinIO’s read ahead technology, we can deliver just shy of an 80% boot IO reduction with nothing other than having our technology in the virtual machine:

Taking a ThinIO averaged test and overlaying it to a baseline averaged test, it’s clear just how much impact this technology can have:

Wrap up:

So there you have it, with ThinIO, a simple, in VM solution, not only can you seriously reduce your IO footprint, boost user performance and achieve greater storage density per virtual machine, you also can also massively negate the impact a booting VM has on your storage.

If you would like a chance to test ThinIO pre-release, find access to the public beta below. Thank you for your time and happy testing!

Test, test, review and tune. Rinse and repeat!

We’ve spent months load testing, tuning, fixing and retesting ThinIO. And for the first time we’re going to start talking about the dramatic results ThinIO can have on storage scalability and user perceived performance.

During our extensive testing cycles, we’ve covered:

• Horizon View
• Citrix XenDesktop
• Microsoft RDS

We’ve been seeing very similar, if not identical results when testing against pools on the following storage types too:

• XenServer SR
• VMFS
• NFS
• Microsoft Clustered Shared Volumes

For reference the statistics we’re sharing today are based on VDI via VMware Horizon View 6, these figures are averages across at least three independent tests. All details of the tests that exported these results are covered below.

Testing details:

The VM’s we tested in this particular workload are as follows:

• Testing Method: Login VSI 4.1 Medium Workload
• Operating System: Windows 7 x64 SP1
• System ram: 3gb
• vCPU: 2
• ThinIO cache: 350mb
• Technology:  VMware Horizon View 6
• Test runtime: 1 hour*
• Statistic sample period: 5 seconds.

With that out of the way, lets jump right in!

Storage IO:

The number of IO’s per second is crucially important when dealing with storage; many, many small IO’s sent to sparse locations on disk are a killer to storage technologies, only made worse by certain file systems.

As a storage acceleration and negation technology, were extremely happy with the IO’s reduction we see on the storage:

Even with just 350 MB of ram as a cache, we achieve phenomenal IO reduction.

Storage MB / Sec:

But IO’s are just one part of the puzzle, what about the size of the data requests being sent to the storage?

A true solution to take the pressure off the SAN, improve user performance, and increase storage density needs to tackle both the IO and the throughput problem.

As you can see above, with just 350mb we’re very good at it!

Side by Side Comparison:

So rounded figures are fine so long as the data is trustworthy, but here’s a real preview laid bare for your analysis.

Here’s a direct comparison on NFS and VMFS of taking a standard load test IO pattern and comparing it to an identical test with ThinIO installed in the VM:

VMFS:

NFS:

As you can imagine, we’re extremely proud of what we can achieve with as little as 350mb per desktop.

The beauty of our approach is simplicity, your users can see this benefit not in a matter of weeks, days or even hours. ThinIO can be up and running in minutes, delivering reduced login times, storage acceleration and providing a far deeper density on your current storage.

Wrap up:

So there you have it, we’ll be adding additional blog posts in the coming days looking at Remote Desktop Services / XenApp, intelligent cache management built in and our Read Ahead technology, so check back. In the mean time, if you would like a chance to test ThinIO pre-release, find access to the public beta below. Thank you for your time and happy testing!

* Eight-hour figures and complete statistics are also available, we have nothing to hide and we’d encourage you to get in touch with the ThinScale team and we’ll share them with you.