What are /r/btrfs' favorite Products & Services?

From 3.5 billion Reddit comments

I switched to Syncthing -- you can run the same sync folder if you symlink a Syncthing directory to where Dropbox expects to find its folder (if you have at least one machine that is extfs).

Also moved to Cryptomator when I migrated to Syncthing (in case you want to the dual service sync thing and maintain privacy).

Given that your system becomes unresponsive when doing this I/O, I'm inclined to suspect this is an I/O scheduler problem.

Have you changed any I/O scheduler settings? If so, perhaps put them back to their defaults. If you're running defaults, then perhaps try the BFQ scheduler:

> Regardless of the actual background workload, BFQ guarantees that, for interactive tasks, the storage device is virtually as responsive as if it was idle.

BFQ will be a useful test since it will confirm for you whether the I/O scheduler is the culprit here. If things are better under BFQ, then you likely want to do your own research and make a more informed choice of scheduler. BFQ might not be the best for your overall workload.

If changing the I/O scheduler doesn't help, then I wonder if running the copy under ionice would help. Something like: ionice -c 3 cp foo bar

Two-way syncing is not possible using btrfs snapshots and btrfs send/receive - the problem is that a received snapshot on one end cannot be used as a base to do an incremental send back the other way. (The reasons for this are kind of complicated)

I personally use Syncthing and it works quite well. I would not recommend syncing your entire /home directory (that will cause lots of conflicts, and you won't be able to have different settings appropriate to each system…) but syncing e.g. your Documents, Pictures, etc. directories makes a lot of sense.

Syncthing can sync between multiple targets - syncing to your desktop, laptop, and an always-on NAS will work well, and you can use btrfs snapshots on the nas to do backups.

> RAID1ing an SSD and a spinning drive

mah, I'd go for a sync-ing mechanism from SSD to HDD if you want to have multiple copies of your data (rsync or unison). Have a look at syncthing to sync stuff across different machines.

> protect from bitrot

btrfs has checksums that already deal with this issue. Bitrot will not be an issue.

> Task 2

KISS: create your new RAID1 with 2* 4TB, copy your stuff, ditch the 3TB drive.

> Finally

EDIT ~~Nope. RAID1 supports up to n-1 disk failures because each disk is a copy of the next one. 3 disk RAID1 supports 2 dead drives.~~ RAID5 on btrfs is currently not stable, avoid it.

Meh, btrfs uses terms incorrectly. grrr

I added 4 spaces to the beginning of every line that was command line output. That puts it in "code" formatting.

This is code mode. 2nd line.

This browser plugin adds a live preview as you type:

https://redditenhancementsuite.com/

The plugin also adds a "source" link under every post, allowing you to see how they accomplished the formatting.

The Linux kernel documentation is pretty thorough, you should be able to find anything you need here:

https://www.kernel.org/doc/html/v4.14/admin-guide/kernel-parameters.html

The code changes over time so replace the version in that link above with your kernel's major version for the right release.

> Huge swaths pretty much need to be redesigned, and that involves sitting back and re-engineering some basics.

Like? If there's so much that needs a ground-up redesign, should be pretty easy to come up with a specific example.

> not a production ready file system.

SUSE seems to think it is. Works great far as I've used it.

> the Bugzilla. You know, the place where the developers ask users to log their bugs.

Except if it isn't the place the maintainers/developers want people to log the bugs and engage in the discussion, as per the kernel "How to file a bug" FAQ

> Little vendor support (RH dropping it) speaks volumes about the current status of the project.

Except Docker, Facebook, SUSE... RH "dropping" support is a non sequitur as they never supported it in the first place, and are making Stratis as a user-friendly wrapper for the normal RH storage stack. This makes sense because RH's business is in keeping their existing customers, people uninterested in changing things that are working, and also because RH had not heavily invested in the code (compared to the work put into XFS). https://www.suse.com/c/butter-bei-die-fische/

All of this is ridiculous anyway, you're just a bandwagon poser who isn't really interested in making or improving things.

What size drive? When you added one did you perform a balance afterwards? That's what takes the time.

I added a new 14TB drive to an existing raid10 setup (4x10TB drives) last week and the balance took 4 days to complete.

All my drives are enterprise sata non smr drives such as this: https://www.amazon.co.uk/dp/B07DHY61JP?ref=ppx_pop_mob_ap_share

>that being said, the only one pressing question I have in this regard that cannot wait for me to figure out by testing is; do you need to have all of the drives that make the "single volume" online / hooked up at all times for this to work?

Yes you do. All disks that belong to a volume need to be attached for the volume to be read/writable. If you're missing a drive it'll only mount in a "degreaded" state, which is read-only if at all. So if you can't have more than 1 disk attached at the same time, maybe not the best solution for you.

>Finally, speaking of cataloging the files in your archive / cold storage, do you happen to have any idea FOSS application I can use to replace the Where Is It software I spoke about earlier?

I think what you're looking for is called "ERP" (Enterprise Resource Planning) software. There's lots of options, but I wouldn't be able to tell you which one is best. A good first stop when doing this is just to google "alternative to <thing>". Here is a list of alternatives for &quot;Where Is It?&quot;, maybe give those a shot.

Strange, I get faster read results on my Sabrent Rocket 1TB (PCIE 3). And with nocow directory it's close to 3GB/s for read and write.

What do you get with fio?

Try these settings:

directory=/mnt/storage0/bench/fio/ filename=fio_test_file

direct=1 buffered=0 size=1g

startdelay=3 ramp_time=3 runtime=5 time_based

ioengine=io_uring force_async=4

rw=read bs=1m iodepth=8

Change the directory and save it to a text file. Then you can run it with fio.eg:

fio read.fio

If you want to use the more common libaio switch ioengine to libaio and remove force_async=4.

Could be a problem with HMB. Try booting your kernel with <code>nvme.max_host_mem_size_mb=0</code>.

Might also try twiddling ASPM/ASPT in powertop and see if that makes any difference.

> I've done this experiment myself,

Well, since you stated this I decided to take a look at swsusp docs:

* BIG FAT WARNING ********************************************************* * * If you touch anything on disk between suspend and resume... * ...kiss your data goodbye. * * If you do resume from initrd after your filesystems are mounted... * ...bye bye root partition. * [this is actually same case as above]

Wow. Hibernation is really broken. I don't use it and I suggest you don't.

I thought that with the addition of the write time checker and systemd doing a full sync before suspend it was enough to ensure no inconsistencies between metadata in memory and on disk. If, at resume, you tried to modify something in the filesystem and the expected generation doesn't match, the filesystem would just reject that transaction and become ro.

EDIT: I even go a step further and I completely disable CONFIG_HIBERNATION in the kernel. Hibernation never worked fine for me.

In addition, you might also need to set ‘dirty_ratio’ and ‘dirty_background_ratio’. These are similar knobs but at the VM layer rather than the file system.

https://www.kernel.org/doc/Documentation/sysctl/vm.txt

Note: I don’t know how the file system knob interacts with the VM level one: https://serverfault.com/questions/945936/how-do-filesystem-commit-interval-options-interact-with-vm-dirty-expire-centisec

> If you're missing a drive it'll only mount in a "degreaded" state, which is read-only if at all

Thought as much, but still I shall keep it in mind for other projects.

>A good first stop when doing this is just to google "alternative to <thing>"

Thanks for the tip, but when I decided I could no loner live with Windows and wanted to come over to Linux, I made a list of Windows based applications that I could not live without and went on a hunt for Linux (FOSS) alternatives before making the actual switch. As mentioned previously of that long list the only remaining one is the Where Is It software, which is the sole reason why I still keep a W10 VM around. Every entry on that list has been replaced with a, in my opinion much better, Linux alternative so far. Since Where Is It is such an important part of my data archival workflow though I have been looking into alternatives periodically ever since hoping for something interesting to pop up. From my research the closest Linux based app, which also runs on Windows as a matter of fact, would be Virtual Volumes View which is also available on that alternative to list you linked. But sadly it is a bare bone version of what Where Is It is at the moment with in my opinion decades of development required to catch up.

Either way, thank you for all the information you provided, this was very helpful indeed.

cheers.

Actually, ceph now have own blockstore "bluestore", not a filestore for better Speed. But you need 3 machines at least today. I wish they could make a homeceph for 1 machines. http://ceph.com/ Obvious reason to Love ceph is combine different sizes of discs and SSD/spinning. No Raid but spread out files on many discs.

You can use btrfs device usage /bucket to check how much data is left on each device.

I usually also use glances to check on disk activity.

I do not think that transfer should be taking that long. Is there any warnings/errors on dmesg?

Good point on the CPU. I didn't test exactly how much more CPU the compress-force option takes. I have a 12 core Xeon in this server so I'm not really concerned about the extra CPU cycles.

Great choice btw going with the AMD CPU. AMD is killing it lately and the price/flop ratio. For AMD CPUs is so much better than Intel. Plus you get more PCIe lanes if I remember correctly.

Also, I feel your pain with the RAM prices -- they are fucking insane right now. However, I did find a good deal for 32GB DDR4 ECC dimms (made by Kingston). Right now they are going for around $300 per dimm which is a pretty sweet deal.

If you want more info on the memory, let me know and I'll be happy to send you a link.

I am waiting for the new 7nm AMD CPUs to hit the market myself.

Edit: Here are some Samsung ECC 32GB dimms for $300: https://www.amazon.com/gp/product/B01DTJ8EU6/ref=oh_aui_search_asin_title?ie=UTF8&amp;psc=1

The speed is slower but I am using them for one of my ES nodes and the performance is great.

Thanks but I've already bought everything. This is an expansion of an older system. Though for cheap+lower power+ECC in a box that's just providing bulk storage I'd definitely go with a Pentium.

Edit: also, these are the 10TB drives I got - they had been on sale for $305:

Seagate Enterprise Capacity 3.5 HDD 10TB (Helium) 7200RPM SATA 6Gb/s 256 MB Cache Internal Bare Drive (ST10000NM0016) https://www.amazon.com/dp/B01DSRHFOK/ref=cm_sw_r_cp_apa_i_Pm-iCbQXKG9K1

It definitely wasn't the cheapest $/GB, but it was in the range, but the 5 year warranty made it seem like a fantastic deal.

Possible workaround: Put /boot on a flash drive.

Both of my home machines have something like this inside and connected to the motherboard's USB header:

https://www.amazon.com/dp/B000IV6S9S

And attached to that is just a regular old USB thumb drive to serve as my /boot. This was originally done so that I could have /boot be on something other than my raid6 array and I wouldn't have to worry about having /boot copied on multiple disks in the array or anything like that. It lets me keep the array's disks simple and disposable. Only one array, only one partition (or no partitions) on every disk.

Since /boot gets very little writing, even cheap USB flash drives tend to last forever. Just dd a full image of the flash drive somewhere safe occasionally just in case it dies. I've been doing this since 2009 or so, and haven't had a flash drive die yet, but I did mysteriously lose a motherboard's header port and had to move that flash drive to be external.

Flash drives with USB header connectors right on them do exist, but I decided it was easier to just use the cheap adapter cable and then be able to use any random flash drive I had lying around. That way, if one dies, I'm not scrambling to find an immediate replacement of an obscure item.

Back when motherboards still had legacy IDE ports that were never used anymore, I used to use this sort of adapter for /boot on a compactflash card:

https://www.amazon.com/dp/B001TIXJ5U