Sorry for my rude title Scott 
but I was just on the phone with me ex Capgemini colleague Ernst Cozijnsen and we had a long discussion about super fast storage for replicas in a View environment. In the end we settled for RAM. I immediately jumped into my lab to setup a proof of concept to see how it performs and it’s pure awesomeness, faster than fast and quicker than quick. You want to see what we discussed? Well here’s the story. In the proof of concept I’m using a Windows 2008 64 bit Enterprise server with 7 GB of RAM and the StarWind iSCSI target software. I’ve configured this iSCSI target with a RAM disk of 6 GB and created a VMFS on the iSCSI target. I’ve cloned a Windows 2003 virtual machine to the newly created VMFS hosted in RAM and did some performance checks. I was astounded.
Wednesday, 15 December 2010
Hey Drummonds forget SSD – RAM is the future
Comments
Display comments as
(Linear | Threaded)
Nice one Eric. yeah, you could have a server with 512GB of MEM and Starwind software (which is way cheaper than storage system), and use that solution with RAM disk for your DB's or high load apps Not that big, but still
Not that big, but still
#1
Marek
on
2010-12-14 22:57
(Reply)
yeah.... rock on! this is the hard proof that simple ideas can be mind blowing.
Imagine running a golden image from such a ultra high performance datastore, it would beat any storage box, especially in price.
The next step: boot from NFS, run in RAM and dedubbed by VMware. This is getting better by the day 
#2
Ernst Cozijnsen
on
2010-12-14 23:16
(Reply)
In 2006 when I was running the Intel team that was developing and testing SYSmark 2007 we purchased a RAM disk for evaluation. We were curious exactly how fast a desktop benchmark would run if we could eliminate pesky drive arms. The results were nothing short of fantastic.
However, one thing we did quickly realize is that traditional busses are designed only to exceed the limits of the devices attached to them. Use something like RAM on PCI and you're really running a PCI benchmark. Nonetheless, it was a lot faster than our desktop rotating disks.
Awesome idea, Eric. Now go buy yourself a big-ass battery to connect to that DRAM.
#3
Scott Drummonds
(Homepage)
on
2010-12-15 01:00
(Reply)
I have to try this out in the lab too!
How does the speed compare to FusionIO?
thanks!
#4
Fletcher Cocquyt
(Homepage)
on
2010-12-15 01:08
(Reply)
RAM has some issues with price per capacity, however if you need to deliver 600,000 IOPS how about a RAM SAN?
Unsurprisingly there is such a thing, http://www.ramsan.com/products/4
#5
Alastair Cooke
(Homepage)
on
2010-12-15 01:44
(Reply)
isn't this also what a huge ass ram cache does on a san? Or am I seeing that the wrong way?
#6
Wouter Kursten
on
2010-12-15 11:43
(Reply)
Great performance and interesting work! But what happens when the power goes out? or the server gets booted.. How do you maintain state? seems to me like an awesome solution for swap disk..
#7
Warning Will Robinson
on
2010-12-15 15:47
(Reply)
That is true - We forget we are working with "memory" which doesn't keep data after a powercycle. Disk type systems will be around for a while but I am interested in what kinds of solutions we can see built around RAM disk for Fault Tolerance. Right now the RAMSan is a pretty penny but this still is a VERY cool lab set-up that I am going to have to try later on today. If anyone is look for the Virtual Ram Drive Emulator you can get it here: http://bit.ly/hY53th
#8
Chad King
on
2010-12-15 17:23
(Reply)
Ey Eric,
Surely RAM beats it all. It is approx. a thousand times faster than flash when you have RAM in the storage box, and is about a milion times faster than convential disks. Even looked at the Sun 7000 series? They make a perfect combination of RAM, SSD and conventional disks thanks to ZFS. I got amazing results from the thing
Check out "Breaking VMware Views sound barrier with Sun Open Storage" at http://www.vmdamentals.com/?p=569
Check out "Breaking VMware Views sound barrier with Sun Open Storage" at http://www.vmdamentals.com/?p=569
#9
Erik Zandboer
(Homepage)
on
2010-12-16 08:40
(Reply)
Not very impressive, in fact. quite slow actually. memory should be able to do speeds in the GB/s range. problems with the software?
Per GB SSD solutions are more attractive comparing with you're performance numbers. (price) I.e. Fusion IO h t t p : / / www.techeye.net/hardware/fusion-io-solid-state-drive-has-a-whole-lot-to-give
#10
eric
on
2010-12-16 09:39
(Reply)
Quite slow? You have to compare apples to apples here. RAM could give you GB/s *inside* a computer system, not through a) host operating system b) iSCSI link c) network driver d) client operating system e) client OS drivers (and some layers I forgot here). It is a great indication of what can be done.
In the end, you'd be using any of these solutions from a shared storage pool anyway, meaning physical interconnects (TCP/IP, FC). So is DRAM going to help at that level? Not much. Is it cool? Yep, Way cool 
And you can write on it forever 
Would be interesting to investigate where exactly the bottleneck lies here (and rid yourself of it). The "how fast can your VM go"-contest is about to begin!!
And you can write on it forever
Would be interesting to investigate where exactly the bottleneck lies here (and rid yourself of it). The "how fast can your VM go"-contest is about to begin!!
#10.1
Erik Zandboer
(Homepage)
on
2010-12-16 15:01
(Reply)
RAM disk isn't an alternative solution to SSD disks. Sure u can reach high speed using ram disk, but in case of power falling you loss all your data. But if you want to show high-speed results on ram disks to the audience of your example, you have to install StarWind on separate PC, use 10 Gbit net cards or you have to do a link aggregation (nic teaming). Here's a link where you can see how intel got 1 mln IOPS using SW on ram disks - http://www.starwindsoftware.com/Microsoft-Intel-iscsi-test. In your configuration, weak point is to take tests inside several VMs. In this case results depend on how VMware will allocate and manage resources.
#10.2
Vadim Nekhai
(Homepage)
on
2010-12-16 15:58
(Reply)
Awesome test. It reminds me a research project from Stanford University called RAMCloud. VMware founder Mendel is involved in the project. Here is the paper: The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM.
http://www.stanford.edu/~ouster/cgi-bin/papers/ramcloud.pdf
Steve
#11
Steve Jin
(Homepage)
on
2010-12-18 07:40
(Reply)
I have calculated the following for a budget system with 24GB DDR3
Mainboard (Intel Socket 1366 with Intel X58 chipset) with 6x DDR3 sloten, approx. 200,- euro
24 GB DDR3 (2x 12GB Triple Channel Kit DDR3-kit), approx. 300,- euro
Other parts (CPU, case, disk, etc.), approx. 500,- euro
total approx. 1000,- euro
ps. How much would it cost in the US?
#12
Johan van den Heuvel
on
2010-12-20 22:11
(Reply)
EMC's FAST is great, but it is WAAAAY too expensive and not practical for smaller shops.
I can implement something that gives most of the performance shown here AND has a persistent storage backend. It can be done with hardware/software available today, but I'm pushing FreeNAS developers for something a little simpler. Please email them and/or comment on my FreeNAS feature request here: https://support.freenas.org/ticket/106
A simplified example procedure to do this:
1) Create a VM as Eric described, export two iSCSI target LUNs
2) Create a FreeNAS 7.2 VM with physical hard disk(s)
2a)Use built-in iSCSI initiator capability to mount iSCSI targets described above
2b)Create a new ZFS volume. Specify your hard disk(s) as your main pool, and specify the two iSCSI based mount points as your ZIL (write cache) and your L2ARC (read cache)
2c)Export your new ZFS volume as iSCSI LUN
3) Attach your ESXi to the iSCSI LUN mentioned above
Voila! You now have a super fast multi-GB read/write cache sitting in front of a persistent disk storage. You can now host an arbitrary number of VMs on this volume and they ALL can benefit from a performance boost. This would be particularly effective in a VMWare View environment where you have a lot of VMs using the same backend storage.
The only "issues" are:
--It takes a while for the read/write cache to be warmed, so you won't get an immediate performance benefit.
--The process is a little more cumbersome than it should be. I'm hoping the FreeNAS guys can eliminate the need for step 1 in the next beta version of FreeNAS 8.x
If you guys want, I can describe the process above in more detail
P.S. Why is the formatting in my comment stripped?
P.S. Why is the formatting in my comment stripped?
#13
Matt Van Mater
on
2010-12-27 21:16
(Reply)
A few days ago I’ve posted an article called “Hey Drummonds forget SSD – RAM is the future” on my website. In this article I’m showing a proof of concept with a recorded demo with some performance measurement on an VMFS with live virtual machines in RAM.
Tracked: Dec 18, 14:38