What is Reddit's opinion of
StableBit DrivePool?
From 3.5 billion Reddit comments
78 reviews of this app found across Reddit:
On Windows? StableBit DrivePool.
As for the drives themselves, I would highly recommend checking out the drives health, because it sounds like one or more drives may be ready to fail.
HD Sentinel or StableBit Scanner may be a good option to check the drives.
http://stablebit.com/CloudDrive
will create a encrypted folder on dropbox & other providers and mount it as a drive on your pc, theres a trial.
I haven't used it myself as my upload is a shitty 100kbps
No 8TB 2.5" hard drives exist, however, 4TB 2.5" drives do. Combine that with with an external USB enclosure like this one (the linked enclosure supports up to 15mm drives, which would be important in this situation), and you'd be set. Use whatever is built into whatever OS you're running (or StableBit DrivePool if you're running Windows) and you have what you seem to be looking for. :)
This is normal, actually. The balancing system isn't "aggressive", meaning that it's not going to balance data unless there is a reason to do so. And technically, there isn't one here. Over time, it will use this drive (new files are added to the disk with the most free space).
However, if you do wish to force StableBit DrivePool to rebalance the data, install the "Disk Space Equalizer" balancer plugin, enable it, and it will rebalance the data to use all of the drives equally.
> So, what NAS like options with disk mirrors are available either on esxi directly...
None, at least with the free license. I don't know enough about what's available with a paid license to speak to that...
> ...or on a VM hosted by esxi? I've read virtualized ZFS don't work too well.
Depends on your hardware. So long as your potential ESXi host supports VT-d, and you have an appropriate HBA that you can pass to the VM you wish to use for NAS duties, you should get along just fine. :)
I have an HP Z800 with 32GB of DDR3 ECC, but my 'NAS' VM is actually WHS2011 based, and I use StableBit DrivePool to create a single point of storage. :)
Honestly, something like really isn't needed. The chance that your data is going to degrade is pretty insignificant.
However, if you're worried about it, then a simple read pass of the raw data would be all that you need. Most testing utilities should be able to do this, without a problem.
StableBit Scanner is a good option, as it does a monthly surface scan. HD Sentinel would also work, as it has similar options (but not as automatic, IIRC).
Both work on Windows 10.
>Also, yes I'm planning on upgrading drive capacity. Red = bad.
Tell me about it. I have less than 10% free. Though, I have 116TBs, and that's 9TBs free, right now... And yes, it's a fantastic problem to have.
As has been mentioned already, StableBit DrivePool may be a great solution for you. It's easy to migrate over to (since it's an NTFS, file based pooling solution, no need to reformat), and easy to manage.
Also, migrating over to a pooling solution would allow you to had most (all) of your data drives, and display a single (or couple of) disk(s) to the system, making data management easier.
If you have any questions about the software, it has a fully featured 30 day trial. And if you have any questions, contact their support.
Skip Storage Spaces and look into StableBit's DrivePool. You can add drives of whatever size / interface to the pool, upgrade a single drive without manually moving data around (just need to make sure you have as much free space in the pool as the size of drive you're replacing), and it can do file / folder duplication (you can have it automatically duplicate to 2, up to the total number of drives in your pool). Combine it with StableBit Scanner, and you can even get notifications of when drives are having issues before they go belly up on you... :)
Pretty much anything that reads from the drives works for that, doesn't even need to check the data.
/u/mmaster23 has StableBit DrivePool tagged in his flair (so do I), but they have another product, StableBit Scanner that performances a monthy surface scan (a sector by sector scan). This should be sufficient enough to "scrub" all of your data, and cause the disks firmware to detect and correct problem areas before they become a problem.
If you're using Windows (and kind of assuming you are, sorry), another option is something like ReFS. It has built in integrity checking for the content. No repair (without using Storage Spaces, and a mirrored array), but still.
Linux has half a dozen different file systems that also handle scrubbing and integrity checking.
Baring that, an actual locally stored checksum (like mmaster23 mentions) is a good solution, as well. QuickPar is one way to do that, but there other solutions that work for this, as well. I've been tinkering with Checksum as that seems to work well.
I use DrivePool and Scanner from http://stablebit.com/ (Windows only)
It lets me pool all of my drives together and lets you select which directories or drives that you want to have mirrored. You can even set it to keep 3 or more copies of really important directories.
I like it because it is the easiest tool to use and it sends me an email if anything goes wrong.
RAID, itself, won't necessarily 'degrade' HDDs any more quickly than non-RAID use.
Since you're using Windows, StableBit's DrivePool may be a better option if you're wanting to protect personal data from loss, and don't really care about the data you're seeding, since it can do per-folder file duplication.
If you do decide to go the RAID route, you could look into an LSI 9208-8i or 9211-8i. They're pretty cheap on eBay (under $100). As for HDD brands, I happen to like Seagate or HGST. My "main NAS" includes three refurbished HGST 3TB HDDs (the HGST link is the exact drive I purchased) and one 3TB Seagate Ironwolf. My backup has three Seagate / Samsung 2.5" 2TB drives and one 2TB 3.5" Seagate Ironwolf. ;)
To move the data into the pool, check out this guide:
http://wiki.covecube.com/StableBit_DrivePool_Q4142489
After that, you may want to install the "Drive Space Equalizer" balancer so it redistributes files:
Definitely.
Just remove the drives from the pool, one at a time, and replace them. StableBit DrivePool should handle this no problem.
Alternatively, you can use the "Drive Usage Limiter" balancer to clear out the drives, prepping them for removal. Once it's empty, remove the drive (this will no longer be necessary in future releases, as we've completely reworked the drive removal code to keep the pool writable while removing disks!)
Removing a Drive from the Pool
If you have any questions, ask here or contact us directly at https://stablebit.com/Contact.
If it's a Windows desktop, look at StableBit's DrivePool product. Creates a JBOD array of disks (pretty much any combination of disks), and presents it to Windows as a single drive letter. You can even remove the drive letters of the individual drives (which is what I do). If you did decide to store anything important on it, it supports folder level duplication (up to the number of disks in the pool), so if a drive fails, the data will reside on another disk. They stay NTFS formatted, so if you decide to get rid of the pooling software, any NTFS capable OS can read them... ;)
I use this for my Plex Media Server, and as a general dumping ground for stuff I "just can't live without", but rarely touch... lol
I'd echo the other replies here -- something definitely isn't right. I know this isn't apples to apples, but here is how my 9211-8i, in IT mode, performs with a StableBit DrivePool pool consisting of three 3TB 3.5" drives (two refurbished Hitachi HUS72403, one new IronWolf).
Considering it's a JBOD array, and it only tested with one of the disks, I think it did fairly well. :)
> A few things to understand, hardware RAID involves using hard drives that are the same size, otherwise you can't do it.
Well, that's not entirely correct. You can, but all drives in the array will only have an amount of space used as that of the smallest drive in the array.
> ...configure your drives as JBOD (Just Big Old Disk) which means they are all individual drives, either way you won't have any redundancy, something hardware raid offers.
Also not entirely correct, depending on the setup. Being the OP is using Windows, he could use StableBit DrivePool, which will not only create a JBOD with his mismatched drives, it also offers user-selectable folder level duplication (duplicating the data up to the number of disks in the array). On top of this, many add SnapRaid to gain a parity disk, though it's apparently not calculated real-time, so it seems better suited to data that doesn't change often.
A better RAID/HBA option for /u/Rehmanpa might be something like this LSI 9211-8i, which is pre-flashed to IT mode (this disables the RAID functionality and makes the card behave like a normal SAS/SATA controller). Down the road, should OP desire, it can have the LSI RAID firmware re-flashed to it, if (s)he desired. It also is less likely to have a compatibility issue that PERC controllers can have on some mainboards (like an AMD based board I had, which wouldn't even POST with my H700 installed, and my Z800 which throws RAM / CPU errors with the card installed). In addition, it's cheaper than the card you linked (currently listed at $72 with free shipping). Of course, this only matters if OP is in the US... ;)
If you're having issues getting good playback from your Plex Media Server from aUSB3 HDD, you're having other issues -- it's not (or at least shouldn't be) the external drive. I used my 5 year old Asus G73jh laptop (which only has USB2! and a first gen i7-920) with a 2.5" USB3 2TB drive while visiting family out of state, and had zero issues with video playback using my Nexus Player (laptop in one room, connected to the wireless network, Nexus Player on the other end of the house, connected to the same wireless network). My movies are generally 10-15GB 1080p (just depends on what is available when I download), and my TV shows are almost always 720p.
That said -- if your case is physically large enough, and if you leave the PC on 24/7, why not let your PC act as the "nas"? If it's running Windows, you could use StableBit DrivePool to pool the media storage drives together, point Plex at the new drive pool, and gain the ability to expand the pool in the future (WITHOUT having to manually move all your files around in order to add / replace HDDs). :)
Back when I had a physical PC for "NAS" use, it was a WHS2011 system with a Core i5-2500 and 4GB of DDR3 RAM. I had five 1TB SATA HDDs in the system, connected to the onboard SATA ports. It used StableBit's DrivePool to pool all five drives into a single "5TB" pool. This let me deal with a single drive letter, which came in very handy since that server was also my Plex Media Server, and had sickbeard and sabnzbd installed (this was before I found Sonarr and switched away from sickbeard). It served the TV/Movie content to two Phenom II X4-based HTPC's via the Plex Home Theater software, and served as a general storage / backup location for all the other PCs I had in the house.
Back in November of 2014, I picked up an HP Z800 with two quad core Xeon processors. I then P2V'ed the WHS2011 box and moved my HDDs / WHS2011 into a virtual machine. I also virtualized every other PC I had in the house, so now I have a single host (which itself has been replaced with a slightly more powerful system just a few months ago)... ;)
No. My 'media server' has always performed double duty as "nas". I use StableBit's DrivePool for a couple reasons. Firstly, when the Drobo was first released, I liked that you could eject an HDD and replace it with a larger one without having to manually move a bunch of data around / rebuild the 'storage pool'. What I didn't like was its pricing (and the initial hardware's SLOW performance / USB only connection / ethernet to USB hack they later released). I was looking for something similar, but substantially cheaper, when I happened upon the original Windows Home Server product and it's "Drive Extender". I loved Drive Extender, and am probably one of the very few that didn't have data corruption issues with it. When Microsoft discontinued Drive Extender, several companies created a similar product, but DrivePool is the only one that I know is still around / actively updated. The primary thing I like about it over other products is that the individual drives that make up the storage pool stay NTFS formatted, so I can remove the drives and read their contents in any NTFS-capable OS (should I need to do this for any reason).
The above wall of text was probably more than you were looking for, but it's the text you got! :P
I'm not sure those that have replied thus far understood what you asked.
You can create a VM under ESXi that uses Windows (pretty much any version), as you're already aware. The only way you will be able to attach your two 3TB drives to that VM is if you use 'pcipassthrough' (or IOMMU, if you have an AMD CPU-based host) with a SAS / SATA controller. On my HP Z800, I have a WHS2011 VM with StableBit's DrivePool pooling three 2TB 2.5" Samsung/Seagate drives. I've used 'pcipassthrough' to attach the Z800's onboard six port Intel SAS/SATA controller to the WHS2011 VM. The VM sees the controller just as it would as if it were the only OS on the host, and as a result, it can see / use any SAS/SATA storage device I connect to it.
You will NOT be able to connect the 3TB drives to the same controller you connect your SSD and expect your Windows VM to see them. In addition, you'll need to make sure both your CPU and motherboard support VT-d (Intel) or IOMMU (AMD). If either, or both, don't, you will not be able to do what you're wanting.
StableBit DrivePool. It can do everything that you want. Some of it may require a bit of work, but it's all doable.
And if you have any issues with it, or ANY questions about it, you should absolutely ask their support.
As for moving files into the pool: http://wiki.covecube.com/StableBit_DrivePool_Q4142489
And for keeping the specific files on the pool: File Placement Rules will definitely allow you to do this.
As for the rest, do you mean Storage Spaces or Dynamic Disks. They are similar but drastically different.
There's a great product called StableBit Drive Scanner that will continually monitor your HDD's SMART data and runs monthly sector-scans of your entire HDD--it warned me ahead of time that my last drive was beginning to fail, and I was able to safely evacuate my data to a healthy drive.
They have a fully-functional 30-day trial you could use to monitor the drive for awhile and see if the software raises any warnings.
StableBit Scanner does both the SMART data and monthly surface scans (as well as file system scans).
Otherwise, check out "alternativeto.net", as they have a good list of software.
Need more information :
When you're parked and desire to watch TV / whatever else, are you doing so via solar, generator, batteries, shore power (or some combination of the first three)?
How much storage space do you need for said pictures / TV shows / movies?
What kind of network, if any, will be in the RV?
What kind of space do you have to dedicate to whatever equipment you ultimately end up using?
I'm sure there are other bits of info we'd need, but the above is a good start. I can tell you what I've done for my mother (and what I'd have done differently if I knew then what I know now). I set up a small J1900-based Celeron based PC with a 3TB HDD and two 1TB HDDs. The system runs Windows 10, and has StableBit DrivePool to 'combine' the three HDDs into a single drive letter. On that new drive letter, I created a TV and Movies folder. I installed Plex Media Server and Plex Home Theater, and configured both to auto-start on login. The mainboard I selected has HDMI out, so we just connect that directly to the TV. She uses the Plex web interface from the laptop that is always sitting next to her chair to start / stop movies and TV shows on the 'server'.
What I'd have done differently -- install a Nexus Player / Roku at the TV, with the Plex app, and leave the PC connected directly to the router (right now it connects via WiFi). This way she'd have a nice simple remote to use to browse the content instead of having to use the web interface (which she loves, but I think the remote would have been better). The laptop always lives next to her chair, and was there before the Plex server was installed, so it wasn't something new we had to add to her daily life... ;)
So -- depending on power availability, I have different suggestions as to how you could set things up... ;)
Shamelessly: Install StableBit Scanner, activate the trial, right click on the drive in question, run burst test. The Burst test reads the same data over and over, so it doesn't generate disk activity. But it's great for stress testing the connection. if it's the controller, it will show up here.
Once you've confirmed what it is, uninstall the software if you want.
RAID is not a backup.
That said, depending on what you're doing / storing, StableBit's DrivePool might fit the bill. It has per folder (and I believe file) redundancy abilities, and is excellent for media storage / playback (which is what many of us use it for). Best thing with it (for me) is that the disks don't have to match (or even be the same interface type)...
My house is an all Windows shop, so I have a WHS2011 system with StableBit's DrivePool, and currently have three 3TB 3.5" HDDs attached. This system is my primary data storage location (currently storing all Paizo Adventure Path .pdfs from my subscription, in addition to the .pdf versions of the hardcover books I've purchased, plus Music / Movie / TV shows, plus I've redirected my "Downloads" folder to a share on the pool). It exists as a virtual machine on my ESXi host (but could just as easily be a physical machine, and actually was until November 2015).
For backup, I built an XPEnology system. It currently has four 2TB HDDs (I'll be adding another soon). If you're unaware, XPEnology is the Synology NAS OS / software, but on a standard PC (kind of how folks install OSX on standard PC hardware). It is used solely to back up the rest of my Windows systems, plus it's a secondary storage location for my TV shows (I don't tend to keep movies around for long, since it's rare that I watch them in the first place, let alone re-watch).
For your situation, I suspect adding a 2 or 4 bay NAS would be ideal, depending on your storage needs.
My ESXi primary datastore consists of four PNY CS1311 480GB SSDs in a RAID5 array connected to an LSI 9208-81. All of my VMs primary boot drives live on this array (1.3TB total, 883GB free as of right now). One of my VMs runs WHS2011, and it has a LSI 9211-8i attached, to which three 3TB 3.5" drives are connected. These three drives are controlled via StableBit's DrivePool, the rest of my VMs access this pool through standard Windows shares, and this is where general bulk data is stored. :)
The specific card I linked is pre-flashed to "IT" mode. It is also available with its normal RAID firmware installed.
The JBOD software I mentioned was StableBit's DrivePool software. It is extremely easy to use, and there should be plenty YouTube videos. StableBit also has a channel, though it only has two videos. They also have a forum, which is available at the DrivePool link I provided.
You didn't mention your OS of choice. If you're using Windows, I'd suggest looking into StableBit's DrivePool. It lets you combine drives of varying sizes / interfaces (or all the same size / interface) into a single drive. It has some file/folder redundancy built in (if you want it), and works well for a Plex server -- this is what I'm using for mine. :)
The way I do it is : I've added an LSI 9211-8i (flashed to "IT mode") to my host. I used 'pcipassthrough' to attach the card to my WHS2011 VM. The VM sees the card just as if it were installed in a physical computer, and thusly, it sees any drives connected. From there, I happen to use StableBit's DrivePool to make them show up as a single drive.
I run mine in a VM, however, unlike the rest of the replies you've received as of this writing, my ESXi host is an "all-in-one" solution, as I wanted to consolidate all the crap I had running in the house, which had the side-effect of also lowering my power usage. :)
My host is an HP Z800. It has two Xeon X5677's and 32GB of ECC DDR3 RAM. The primary datastore is housed on an LSI 9208-8i RAID controller, and consists of four PNY CS1311 480GB SSDs.
The "plex" VM is WHS2011 based, and has four vCPUs, 6GB of RAM, and a 160GB vHDD. Plex Media Server, Sonarr, and sabnzbd live here. I primarily use two Nexus Players to view the content stored in my Plex Media Server, so there is rarely ever any transcoding going on. I rarely have to transcode, and when it does happen it's only ever one stream, and that's only when I'm not home (which is rare). The media is stored on a StableBit DrivePool, which consists of three 3TB 3.5" HDDs connected to an LSI 9211-8i flashed to IT mode (pcipassthrough connects it to the VM directly).
How many streams, and how many of those need transcoding, will pretty much dictate what kind of hardware you're going to need for a host. :)
I'm doing something similar to what you are wanting, save for my guest OSes being all Windows. The part you're missing is that, for your NAS VM, you want to have a SATA controller / HBA to dedicate to that VM via pcipassthrough.
In my case, I use an LSI 9208-8i for my ESXi datastores, which frees up the two SAS/SATA controllers built into my mainboard, which I passed to my WHS2011 VM (which acts as my 'NAS', Plex Media Server, and media grabber (Sonarr & sabnzbd)). I've attached my bulk storage drives (currently two 2.5" 2TB and two 3.5" 3TB drivs) to the onboard SAS/SATA controllers, and WHS2011 sees them just as if the only OS on the host was WHS2011. I use StableBit's DrivePool to provide a single pool of storage (with configurable redundancy), and StableBit's Scanner for drive / S.M.A.R.T. monitoring and notifications (which can be done via email, SMS, tweet, speech to text, and something called pushover).
With what I've described, there isn't any reason you couldn't continue to use the Ubuntu setup you were using prior to your exploring, FreenNAS, unRAID, etc instead of Windows.
Does it Blue Screen after it freezes? I would check the windows crash dump.
Additionally, I would also run MemTest and see if you have any memory issues in safe mode. Also I would try to run some anti-malware like Malware Bytes just in case and scan for rootkits.
And also you should run StableBit Scanner to see if your HDD has any SMART errors.
When it BSODs, check the crash dump. Download BlueScreenView and see what it indicates the issue is.
Google/Bing from there to see what it means.
Also, "weird issues" tend to be memory related, so do yourself a favor and run a memory test. Also, check the system disk ("chkdsk c: /f").
Also, how's the hardware health? Have you tried StableBit Scanner or HDSentinel or the like to see if your system disk is "healthy" (eg, no SMART errors)?
In my 2.5" R710, the 8 bays are separated in two groups of four, and each group has a mini-SAS cable that runs to the PERC/6. Yours may be similar (but in groups of three).
If that's the case, it won't be optimal, but you can disconnect one of the two groups from the PERC/6 and connect it to a second controller (which you'll have to purchase, along with a proper cable). At this point, you'll have a controller you can attach to your NAS VM (FreeNAS, unRAID, OMV, RockStor, whatever), and any drives attached to that controller will be seen by the OS without having to go through the 'create a datastore on each WD and assign it to the VM' shenanigans...
This is what I'm doing in my HP Z800 -- the datastore disks (four PNY CS1311 480GB SSDs) are connected to a LSI 9260-8i in a RAID5 array. My 'nas' disks are connected to the onboard Intel six port SAS/SATA controller, and I used 'pcipassthrough' to assign that controller to my WHS2011 VM. The three 2.5" 2TB drives attached to this controller show up in the OS just as if it were a physical PC. I use StableBit's DrivePool to present the OS / my network with a single drive letter.
First, the 100% disk usage in Task Manager means literally nothing. Ignore it. 100% can mean 5kb/s or 500mb/s. It depends on the short period of time it's monitoring.
If you want to see what's really using your disks... use "resmon". Open the disk tab. That will give you a good idea.
As for disabling Windows Defender, Open Settings, Update & Security, and "Windows Defender". Disable the real time scanning and scanning. But install something else. That or run "services.msc", find the "Windows Defender" service, and either stop it, or change the startup type to "manual" or "disabled".
And running "CHKDSK C: /r" would be a very good idea.
Also installing software like StableBit Scanner (trial-ware, $30) or HD Sentinel (Premium-ware) would be a very good idea for reading SMART data from the drives (and in the case of StableBit Scanner, preventative maintanence).
ESXi is not an OS, it's just a Type 1 hypervisor (It runs virtual machines, that's all it does). You would have to install a virtual OS within ESXi.
Then inside the virtual OS is where you would do the drive pooling.
But if you are only installing 1 OS on a PC then there is absolutely no reason for ESXi.
The only reason for ESXi is for running 2 OS on one PC at the same time.
If you want to stick with Windows there are drive pooling options too.
The best options are DriveBender:
http://www.drivebender.com/
And DrivePool:
http://stablebit.com/DrivePool/
They both have trials, try em out before you buy.
I still recommend having a dedicated drive, even a small one for the OS and then the pool will be separate. You don't want to install your OS to a drive pool. It's usually not even possible.
If you want an easy RAID solution in Windows (NTFS), and I am talking the absolute easiest setup and maintenance then you'll want to look into Stablebit Drivepool and Scanner.
I'd highly recommend you read their entire FAQ but in short you have the ability to add any drives of any size/mfg to a pool. Their software manages the array for you and balances/duplicates all your data. Adding and removing drives is a breeze. Also, if you have an OS failure, you can simply reload Windows and their software and all your data is there. You can literally move your drives to any computer and still have your data intact, it does not rely on a hardware controller.
I have a setup with a 320GB OS drive and 5 other drives ranging from 500GB to 750GB. My drives are leftovers and I've had two fail (separate times) but I've had no issues with data whatsoever. I test drove it for a while keeping my 2-bay RAID 1 NAS online but eventually decided to move over to Drivepool full time.
Performance is not an issue. They're in an old Core 2 Duo machine with 4GB. I have a gigabit NIC and I can stream 15GB video files no problem. Speeds are good over a gigabit network
You could start with a couple WD Red 3TB/4TB and expand down the road. TeamViewer is everyone's weapon of choice for free remote these days so I'd stick with that.
I guess that wasn't in short, but you get the idea. Check out their forums for guides and addins as well. It's a small but very active community.
Quick edit: Their Scanner software keeps an eye on the health of the drives and has an email alert system the lets you know if one of your drives has an issue (sector, SMART, Temp etc)
Stablebit Drivepool
Does everything the OP asks for. Ive been using it for 8 years with 0 lost files or corrupted files. It could care less what size the drives are or how they are connected.
>There is no official support for Plex on Server 2016 so there's that
No, but they support Windows 10, and it is supposed to be the same kernel. So you should be fine.
>Microsoft not helping things with removing Essentials from Server 2019
RIP Windows Home Server. :'(
I'm 100% biased here, but you could use StableBit DrivePool and SnapRAID for the storage.
> I have considered FreeNAS, UnRaid, etc.. But I really do not want to go buy ECC hardware.
None of this requires ECC capable hardware. If you want as high a guarantee as is possible against data corruption / bit rot, ECC certainly helps, but it is far from necessary.
> I have a new 8th gen i5 Pentium G5400 system I could turn into a NAS box if needed.
That is absolutely viable (even if "i5 Pentium G5400" doesn't make sense, since those are two different processors). If you are comfortable with a Windows environment and don't like the idea of / need RAID for uptime availability in the event of a disk failure / are comfortable with JOBD, you should look into StableBit's DrivePool. It's not free, but it's far less expensive than a dedicated NAS device and works quite well (I've been using it for years now).
Yes, correct. But when I say "probably", ...
http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Balancing%20Plug-ins#Default%20Plug-ins
StableBit Scanner
This plug-in is designed to work in conjunction with the StableBit Scanner version 2.2 and newer. It performs automatic file evacuation from damaged drives and temperature control.
Volume Equalization
This balancer is responsible to equalizing the disk space used on multiple volumes that reside on the same physical disk. it has no user configurable settings.
File Placement Limiter
This plug-in lets you designate which disks are allowed to store unduplicated vs. duplicated files. It doesn't do anything unless you change its settings to limit file placement on a disk.
Prevent Drive Overfill
This plug-in tries to keep an empty buffer of free space on each drive part of the pool in order to facilitate existing file expansion.
Duplication Space Optimizer
This plug-in examines the current data distribution on all of your pooled disks and decides if some data needs to be rebalanced in order to provide optimal disk space availability for duplicated files (see About Balancing for more information).
The only two likely to cause this to happen are the StableBit Scanner balancer, and the Prevent Drive Overfill.
So, I say "probably", because it may happen. But the likelihood of it happening .... really is as likely as a drive starting to fail.
Do you have Real Time Duplication enabled? link
If not, then this is normal.
If you do have this enabled and you're seeing this, then something is wrong.
In this case, download the latest version (2.2.0.926), and see if that helps. If not, then head to https://stablebit.com/contact so we can help you more directly.
Yes, Read Striping requires that the files in question be duplicated.
If they are not, the file doesn't benefit from this, since we store the actual files on the disk and there is no way to optimize that read.
http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Performance%20Options#Read%20Striping
> I am definitely interested in hearing more.
Since I already had two E5-2670v1's, I found a T7610 barebones (with one HSF) for $445, and I added a second HSF for $35. The rest of the parts in my T7610 came from my HP Z800 (which was my previous ESXi host), which is another system you can look at that will likely be less expensive (since it's socket 1366 as opposed to the T7610, which is socket 2011). If you don't need a high core count / dual CPU system system, Dell and HP both make single CPU systems that should be cheaper than the T7610. Being in Canada, I don't know what kind of system availability you've got up there... ;)
My T7610 has 32GB of DDR3 ECC RAM (which I got on eBay cheaply), an LSI 9208-8i, to which I've connected four PNY CS1311 480GB SSDs (this is my primary VM datastore). All the hardware I mention throughout this wall of text isn't something I went out and picked up all at once. I added / upgraded over time (for example, my 4x480GB SSD datastore started out as two, then three 240GB SSDs, and I went from 16GB to 24GB, then to 32GB of RAM).
The following VMs run 24/7. I spin up other VMs whenever I get bored and decide I want to try something. :)
- Win10 -- 4 vCPU, 8GB vRAM, 250GB vHDD. Radeon RX480 and a USB3 controller are attached via pcipassthrough. This VM is the I use for general purpose use + my light gaming. It has a keyboard / mouse / monitor attached, and I interact with it just as if it were a physical PC.
- WHS2011 - 4 vCPU, 6GB vRAM, 160GB vHDD. This VM has an LSI 9211-8i attached via pcipassthrough, and it has four 3TB 3.5" HDDs attached. This VM is my Plex Media Server, acquires my media (Sonarr, Radarr, sabnzbd), and acts as my "nas" (StableBit DrivePool).
- Win7 - 2 vCPU, 2GB vRAM, 50GB vHDD. This VM only handles my UniFi Controller.
Normal, can be ignored/cleared.
That said, you have it set to balance immediately, but don't have the "not more often" setting enabled, right?
If you enable that "Not more often" option, it should only generate these warnings only on that interval.
>I really wish Drivepool was aware of parity in some form, especially when doing drive balancing.
Why?
Aside from the StableBit Scanner and possibly the "Prevent Drive Overfill" balancers, they won't move data around often, if it all.
You can read about each of the balancers here:
http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Balancing%20Plug-ins
But basically, it shouldn't move the data around often, unless there is "an issue".
Since my "NAS" is actually a WHS2011 VM, I use StableBit's DrivePool product for what is essentially a JBOD array. As a result, if one drive dies, I just lose whatever was on that drive. The nice thing, however, is that I can select individual folders and have their contents duplicated to one (or more) of the drives in the pool, creating some redundancy. :)
There is a huge list of software that will do this.
StableBit Scanner, HD Sentinel, and many others.
You can find a good list on alternativeto.net, which is a fantastic resource for finding software. No affiliation, I just use it a lot.
It's transparent. For instance, if you have "Folder 1" duplicated", it will appear as "Folder 1" on both disks. But on the pool itself, it will show just a single folder.
Features like "read striping" will try to optimize read performance, by reading from both disks (actual striping), will try to read from the "less busy" disk, or will actively avoid cloudDrive disks for local disks.
http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Performance%20Options
Basically, we want the files hosted as closely to how they appear on the pool, and prefer to avoid "trickery" whenever possible. And we let the driver itself decide which is the best copy to read from.
That said, writes are done in parallel, to "all destination disks", so there is no danger period where it hasn't replicated the data to another disk. File modifications are handled the same way, as well.
And this is why we recommend/require multiple SSDs for the SSD optimizer, actually.
Since my storage pool is based on the StableBit DrivePool software, I leave ~%10-15 more than the smallest disk in the pool (assuming I have no more available drive bays / SATA ports). This way, when I click the "remove" button on afore mentioned disk, the pool will contain enough space to hold the data on the disk that's about to be removed, so it can be replaced with a larger disk.
For Windows, there are plenty of tools to do this. Pick yours. Though, handbreak to transcode media may be a good tool, especially if this is going to be used for media.
As for disks, full formats and then scan them with whatever tool ( HD Sentinel,StableBit Scanner or the like.
For either, you can use Alternativeto.net to find a fantastic list of software that can handle this stuff.
Are you using Windows?
If so:
- StableBit Scanner
- HD Sentinel
- and other tools.
They (can) do periodic scans of the media, as well as monitor SMART data. Perfect for what you want.
SpinRite can work, but it's an offline tool, rather than a software package.
I've done it, but with hardware a few generations older. My host is an HP Z800 with two Xeon X5677's and 32GB ECC DDR3 RAM. The host OS is ESXi, the primary datastore consists of an LSI 9208-8i with four 480GB PNY CS1311 SSDs.
My "nas" VM uses WHS2011 and has an LSI 9211-8i passed to it, to which I've connected three 3TB 3.5" drives. I use StableBit DrivePool for the bulk storage, and this VM also serves as my Plex Media server with sabnzbd / Sonarr / Radarr.
My "gamng" VM is Windows 10 based, has an XFX RX480 8GB (the one with the snazzy snap-out replaceable fans) and a USB3 card attached.
The Z800 lives in a closet in my spare bedroom, and I use some cabling from Cables2Go (which are now discontinued) to bring the HDMI / sound to my monitor in my living room. I use this guy to bring the USB signals to my keyboard / mouse.
That said, I recently built a secondary storage server, as I had a power supply failure that took out every single storage device in my system (and fried either / both the motherboard and processor). Luckily the add-in cards survived (somehow). So now I use my 'nas' VM, and I make sure I have anything important to me also stored on the secondary storage server. It was built with a cheap mainboard / third generation i3, 8GB of RAM, four 2TB HDDs, and XPEnology.
> you don't disable balancing in DrivePool, files get moved around across drive.
Wrong. Empirically wrong. There is a rather low chance of this. The exceptions are if you have a damaged drive (StableBit Scanner balancer), if one or more drives is very full, as in 90+% full (Prevent Drive Overfill), or if you're using duplication and there is a large amount of "Unusable for duplication" space (Duplication Space Optimizer).
Otherwise, there is little "chance" that the files will get moved around.
http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Balancing%20Plug-ins
Windows 7?
Check out StableBit DrivePool. It's not RAID, but it's drive pooling. Software based, and stores the data on normal NTFS volumes (so you can read them in any system that reads the disks). Can be moved to new hardware as long as the new hardware reads the disks fine.
There are alos other solutions (Drive Bender, FlexRAID, SnapRAID, etc).
If what you have is working for you, but you're in Windows and tired of all the drive letters, there's always StableBit's DrivePool. Much cheaper than a NAS (either pre-built or custom built), and lets you keep using the hardware you're already using... ;)
For IP camera software, I recommend BlueIris. I use it personally, and it's pretty fantastic.
For the storage, have you checked out something like StableBit DrivePool?
It's generally a very bad idea to use Dynamic disks. There are all sorts of issues with doing so.
If you want just a simple array, check out StableBit DrivePool. It's not free, but it will let you create a simple JBOD type array out of the disks.
I see nothing in your list for the ESXi datastore. Also, unless those six SATA ports are on two different SATA controllers, I see no HBA for your two 2TB drives (for FreeNAS / unRAID / whatever 'nas' OS you opt for).
What (I assume) most people do, and what I've done, is as follows :
One HBA (either onboard or an add-on card) + some sort of storage (I used four PNY CS1311 480GB SSDs in RAID5 connected to an LSI 9260-8i for ESXi datastore). Normal spinning-rust HDDs would work for this, too, but would obviously be slower.
Second HBA (either onboard, if you didn't use it for the datastore, or a second one if your board has two separate SAS/SATA controllers like mine does) for your 'nas' OS, to which you'll connect your storage drives. This controller will be connected to the 'nas' VM via "pcipassthrough". You'll need to make sure your motherboard / CPU support VT-d for this to work. I'm currently using two Seagate 3TB (one "desktop" removed from an Expansion+ USB enclosure, one IronWolf), slotted in two slots of a four slot hot-swap cage, connected to my SuperMicro X9SRA-O's six port SATA controller. My 'nas' VM is actually a WHS2011 VM with StableBit's DrivePool.
As for using Windows as a NAS -- I currently have a WHS2011 VM with three 2TB 2.5" drives for my "NAS". I use StableBit's DrivePool for the actual storage part. The HDDs are attached to my SuperMicro X9SRA-O's onboard six port SAS/SATA controller, which was attached (via pcipassthrough) to said VM. I've been running this setup since November of 2014 (though I've swapped from five 1TB 3.5" drives over to the 2.5" drives about a year ago). :)
Why use Hiren's..... DISKPART can zero out the drive from within Windows....
But a CHKDSK pass would be a good idea. As well as the memory test.
/u/MandiSmash may also want to install a disk utility such as HD Sentinel or StableBit Scanner to check the drive in question. See if there are any issues with it.
Open up an admin command prompt, run "mountvol", and fnd the disk in question.
Then run "chkdsk \?\volume{xxxxx} /f" for the disk in question.
Also, it may be worth installing something like StableBit Scanner or HD Sentinel. Make sure that the disk is healthy.
Yup. If you have small drives, it's no problem. And if SpinRite worked on those size drives...
As for online scanners, there are a bunch, but I use StableBit Scanner because of the integration with StableBit DrivePool, and the simplicity of using both.
No, it's not.
100% is a misnomer and a horrible metric, and Microsoft needs to remove it.
100% means over the last sampling period. That means that 100MB/s can be 100% and 10kbps can be 100%.
What am I saying? Ignore it. Just fucking ignore it. It means literally NOTHING.
That said, check "resmon" for the disk activity and the disk queues (or use something like StableBit Scanner). These indicate the actual activity on the disk. Both are important, and queues are arguably more so, especially with spinning drives.
Not true, at least with DrivePool, unless the drive it selects to accept data you're writing happens to be said drive, which won't always be the case. In fact, StableBit has a plugin called SSD Optimizer for DrivePool that will let you designate an SSD "cache" drive, which will accept all new files being written to the pool, which would pretty well eliminate this issue. :)
You might also consider Stablebit's Scanner. They are the company that came out with Drivepool the WHS Drive Extender replacement.
I need to play with Sentinel again. For some reason I stopped using it a while ago and don't remember why.
Check out StableBit Scanner. Monitors SMART data, can send you notification (email, text, mobile), and performs a sector by sector read of all the data, as well as a file system (chkdsk) pass monthly.
Also, Alternativeto.net is a great resource: http://alternativeto.net/software/crystaldiskinfo/
My setup is like this. I'm running Windows Server 2008 R2. I have it setup with 3x1TB Reds and 3x3TB Reds that run through an IBM1015 HBA card that has been flashed to IT mode. I also have two SSD's that run the OS and essential programs that run off the board.
I then use DrivePool to create a virtual drive through WS2008. DrivePool > Storage Spaces. In my experience it is a great piece of software that utilizes great implementation for storage. It isn't perfect by any means but it's easy to use and great for plug and play. Want to add a drive to the pool? Just plug it in and select the drive to be added to to pool and it goes. It has email alerts and plugins at show you many different pieces of info.
Just food for thought.
Depending on what OS you're running or want to run there are non-raid options that allow you to do what you're looking for.
I use StableBit DrivePool on my server but there are other systems like freeNAS, flexraid, and unraid.
Haven't used anything but Drive Pool but it allows you to add any sized drive you want whenever you want.
> Once the storage space volume is created, can new drives be added?
Yes.
>If the 2tb fails, can I replace it with another 5tb?
Yes.
> Will the drives in the storage space volume be readable by another machine if they are pulled out?
Yes (you may have to set up storage spaces on that machine first, though).
> Are there risks of corruption of the whole array like you see with RAID?
Not really but you never know.
> Is this a good idea at all or should I keep things simpler since my array is so small?
I recommend http://stablebit.com/drivepool instead of Storage Spaces for sure. That, combined with StableBit Scanner allows you to know if your hard drives have any imminent failures and require replacing.
Your best bet would be to use: http://stablebit.com/drivepool
Storage Spaces can do what you want but your requirements in terms of using disks in another computer etc can be done with SS but its simpler with drive pool.
Are there any official announcements about this?
I am planning to remove my WHS 2011 server and add all of its HDDs into my Windows 10 PC.
I have been using StableBit DrivePool on my WHS 2011 and I was going to continue to use that on my Windows 10 PC just because of the bad Storage Spaces reviews which where made back in the Windows 8 days.
With the Windows 10 release I did not read any news about this feature and so I had the impression that Microsoft already moved on and Storage Spaces won't get any improvements anymore. 😥
I don't believe that the files are split at all, so I think you only lose the files on that drive. If the computer fails itself, the files are still available in a hidden folder on the drive and they are there in their entirety, so you can copy them over to something else if needed. Think you lose the file structure, but the files are there.
May add a little more detail for you:
http://stablebit.com/DrivePool/Features
- If something goes wrong with your computer, you can simply plug in any pooled disk to any computer that can read NTFS and all of your pooled files that are stored on that disk are fully accessible under a special hidden folder.
- Alternatively, simply connect all or some of your pooled disks to any other computer running StableBit DrivePool and that disk will instantly show up as being part of that pool.
nope you wont lose any data on your drive, you can store files on the drives separately while the drive is in the pool.
http://stablebit.com/Support/DrivePool/1.X/Manual?Section=Adding%20a%20Drive%20to%20the%20Pool
Drivepool is pretty easy to use. $19.95 but you can try it for a month for free to see if it fits for you.
edit: Oh yeah and if you have Windows 8.1 you can just use Storage spaces
FreeNAS recommends a minimum of 8GB RAM. The general rule is 1GB per 1TB so you would need 12GB for the best performance.
I can't say how well it would work with only 4GB but there's no harm in trying it. Quickly set it up and run some benchmarks. If you're not doing any heavy data transfers you might be okay with 4GB RAM.
There are other options you might want to look at: Windows Storage Spaces, Flexraid, DrivePool
I've been using Storage Spaces for years and I haven't had a single problem with it. It's easy to use, reliable and fast enough for my needs. Supports single and dual parity- think RAID5 and RAID6.
I've always wanted to try Flexraid but never got around to it. DrivePool also looked good but last time I checked it relies on file duplication rather than parity so it's less storage efficient if you want some redundancy.