What are /r/DataHoarder's favorite Products & Services?

From 3.5 billion Reddit comments

It's a bit pricier, but I go with my own NVMe drives inside of a USB C NVMe enclosure for moving data between places or as temporary storage such as for scratch disks. The speeds are much faster and the reliability has been excellent. This external SATA 512GB SSD costs $80 USD for example.

There really is not a need for that since we run 10G speeds on https://the-eye.eu/ servers so it's not hard for us to ingest data at full speeds available to the person sending us data. We also have access to a lot of storage given to us by our gracious donors so realistically it's not an issue for us. Just enjoy hoarding what you find important to you on your own disks and bandwidth.

170 for 2x8TB as well

You can use a tool like photorec that can recover many multimedia files from ext2/ext3/ext4.

If the files were any of this huge list of formats it's likely to be able to recover them.

Posting this top-level so others can see it:

A "trick" some pro/semi-pro photographers use when they don't want expensive goods to "get lost" by the TSA: Use a hard pelican case with non TSA approved locks and include a flare gun with it. Basically you're checking a "gun" that also happens to have your other stuff secured in the same box. The caveat is that when you check your bag you need to declare a "firearm" and process it accordingly (which may or may not have additional fees depending on the airline); but once it's sealed at the ticket counter, TSA is not allowed to touch it.

WHOIS data has imgbox and imagebam as both being operated by "Flixya Entertainment, LLC", while eroshare uses anonymous domain registration. https://sendvid.com/ is another Flixya site which is also going down.

It's entirely possible that flixya operated eroshare as well, on the down low.

If you don't have a motherboard that supports the asus card, you can still buy the bare high point card for $400 and fill it yourself for less than a third the cost of the 4TB preconfigured high point.

They're old drives with likely high miles on them, but the pricing isn't crazy and the drives are generally regarded as decent.

A new 3.5" 4TB is $85, so I'd base your decision off that.

Also, 4TB is what I consider pretty small. You're going to need multiple drives and getting larger disks may save space, power and money.

Create a script that calls wget with all the appropriate mirroring options so all he needs to do is supply the URL. And even that can be prompted for.

Here are some other options: https://alternativeto.net/software/httrack/?license=opensource&platform=linux

> Anyone know of a backup solution for UnRAID that doesn't involve rsync/chron or otherwise usage of the terminal?

You're running Linux. You're not going to be able to avoid the terminal forever and you're severely limiting yourself by not using it. People just don't really write GUIs for Linux that often because the people who are able to write them have no motivation to do so.

Duplicati can run on Linux and has a web interface but you'll likely need to use a terminal for the initial set up.

I'd really recommend something like a snapshotting filesystem, Attic or Borg though. All of which will require use of the terminal (unless you switch to FreeNAS, which can configure periodic snapshotting and replication through the web UI).

Saw this earlier. You can download the 7z archive here: https://haveibeenpwned.com/Passwords

The passwords are stored as SHA1 hashes so they can't just be used for bruteforcing. Instead, it's meant for website owners to hash a user-entered password and see whether it has been Pwned at some point and prevent them from continuing.

> Does https://www.softwareheritage.org/ has a copy of it ?

Yes (as can be seen from your last link), but the last snapshot was last december.

But this is Git, so the developers most likely have an up-to-date clone of the repository on their computers.

This is my favorite representation of a TB.

https://www.theverge.com/circuitbreaker/2019/2/25/18239433/1tb-microsd-card-sandisk-micron-price-release

https://www.amazon.ca/SanDisk-MobileMate-microSD-Card-Reader/dp/B07G5JV2B5/

• Kid3 - audio tagger
• ffmpeg - multimedia framework for transcoding, remuxing, etc
• opus-tools, flac-tools, etc - deal with these formats in more advanced ways (encoding, querying info, etc)
• GNU Parallel - run the above commands in parallel to take advantage of your multi-core cpu
• Nicotine+ - FOSS soulseek client
• qBittorrent
• Mullvad VPN - probably one of the most trustworthy VPNs out there, a necessity for the above two.
• A Linux distro - in addition to just being better OS, most of the hoarding toolkit works best on native Linux
• aria2c, wget- download in bulk
• Music Player Daemon - enjoy your music collection
• mpv - enjoy your video collection
• ranger and vim - bulk file renames
• shntool - split audio based on cue files
• tar and zstd - the best archival and compression formats, respectively
• rsync - copy data from one place to another, such as via ssh

Can you elaborate on this? Are you referring to <code>youtube-dl-aria</code>, because that's appaerntly obsolete?

Or are you talking about aria2? And if so, how does that relate to YouTube downloading?

this really needs to be emphasized. backups can give a false sense of security. need to have a process in place to restore the data. GitLab learned this the hard way recently.

Agree, have a 128GB and 256GB and no issues in the couple years I've had them. Cheaper and more portable than other name brands

I wish the uploaders spent a bit of time explaining how to consume the data. I often see interesting stuff uploaded to academic torrents, with literally zero description of how to ingest/use the data shared, or context of where it came from (date ranges, sources, reason for sharing).

As an example: http://academictorrents.com/details/0a853fdcc1d28c306d75e29195a5536087f6e2b4

I know it's street map data, using the PBF format. but how would you use it? Do you need ArcGIS? Is there a freeware tool to convert it to other formats? Uploaders take note, context is very helpful to get people interested in your uploads.

We would use these at my last job for virus/malware removal. They have a physical lock switch which prevents writing to the drive on a hardware level. Kanguru Solutions 8GB Flashblu30 https://www.amazon.com/dp/B00JJIEHJE/ref=cm_sw_r_cp_api_i_VmKZCbVPCQRKQ

We made them bootable with a program called SARDU. Last time I looked it wasn’t being kept up to date.

Each Snowmobile includes a network cable connected to a high-speed switch capable of supporting 1 Tb/second of data transfer spread across multiple 40 Gb/second connections. Assuming that your existing network can transfer data at that rate, you can fill a Snowmobile in about 10 days.

https://aws.amazon.com/blogs/aws/aws-snowmobile-move-exabytes-of-data-to-the-cloud-in-weeks/

Minor correction. It's Mullvad. As for the reasons:

1. The consensus in certain tech circles is that they offer better privacy (no one can verify though but when multiple people recommend when they don't have to, it might be worth something)

2. They are the service provider for Firefox's own VPN service (FF acts as a reseller in this case)

3. You don't need to give them ANY personal information to get an account. Hell, you could mail them cash anonymously.

How are you with python?

https://mitmproxy.org

Wouldn’t take much effort to find anything with a header of “content-type: video/mp4” or whatever it is we’re talking about here, and have python stash the response body somewhere. I’ve used this method for saving all kinds of stubborn shit. All the JavaScript trickery in the world isn’t gonna stop this.

Automate requests through the proxy with selenium, maybe?

Whether it's hard drives or cans of herring, Amazon just doesn't give a damn. In the latter case, the manufacturer first used a very heavy duty plastic shrink wrap, then bubble wrap to protect the stack of 9 cans. Amazon threw it in a box with no added padding, resulting in dangerous dents to the top and bottom cans; they quickly issued a 2/9th's of the cost refund rounded up to the nearest whole dollar ... but scored it as a "goodwill" refund, i.e. as far as I can tell, internally declaring it to be my fault, which one day might help result in an account termination.

As long as Newegg is still carefully packing the bare drives they ship, I see no reason to ever try buying one from Amazon.com in the US.

>frugalmoney.xyz

Yeah how about no.

Here's a link without the aids.

-Edit-

Just checked and this account posts tons of links to sketchy websites.

> 100TB drives in 10 years

In a science fiction short story I wrote in the early 80's, I posited that someday we will have all the music there ever was stored on a wristwatch that responds to voice commands:

> Play "Spirit in the Sky" by Norman Greenbaum

and plays the music in tiny speakers surgically-implanted behind our ears.

Now that's pretty much a reality, for only $25! The speakers would be Bluetooth earbuds instead of implants, but it's close.

Storage is limited to an SD card, maybe 256GB. When I wrote the story, I had about 8 linear feet of LPs and a couple hundred audio cassettes. Wild guess: as MP3s the would occupy 25GB? I have a lot more than 256GB of music now, mostly because of a psychological problem with hoarding, but the music I actually listen to nowadays would probably fit on a 256GB card.

When they're selling 100TB drives they'll also offer 10TB SD cards; then my whole collection will fit in my watch.

This is nothing really special.. i'd rather buy a 4TB drive on amazon for $20 more and be 100% I have warranty on it to be honest.

Says $10 per month, per device at this link https://www.crashplan.com/en-us/business/

It says unlimited. I think I'll pass though, if I add another 10TB or so to my backup the client is likely to completely break anyway assuming crashplan pro has memory issues.

> Case in point, I am considering using the service for remote backups, but would want to retrieve the majority at once in case of need... Now I need to redo my sums ;)

You should consider the Infrequent Access Storage Option on S3. It's somewhere between S3 and Glacier:

• Price is $0.0125 per GB (Glacier is $0.007 and S3 is $0.030 per GB)
• You don't have to wait hours to retrieve your data.
• You just pay $0.01 per GB more for retrieval than on S3, request pricing is also a bit higher.
• You pay for a minimum 30 days of storage when you upload something, even if you immediately delete it. Also files smaller than 128 KB are treated like 128 KB files. So your backup program should merge small files into larger archives.

The main good thing about Infrequent Access Storage is that it's not as complicated as Glacier and easier to calculate.

Berkeley had to take down all their videos in 2017 because of a ruling on the Americans with Disabilities Act, where the judge said they had to provide subtitles or else take them down. Is that what you're thinking of?

https://thenextweb.com/insider/2017/03/17/berkeley-university-deleted-lectures/

Those things are ancient. I had pretty much the same model 10+ years ago.

Personally I've had good luck with the https://www.amazon.com/iStarUSA-Group-3x5-25-Trayless-BPN-DE350SS-BLACK/dp/B008RRZI14. Tool-less, SAS/SATA, better airflow and you can find it on sale for $40-50.

It’s a bit on the pricey side but seems to be pretty well built. Fan is a bit loud on high but if drive temps stay good I may test dropping it down to low.

https://www.amazon.com/dp/B00DWHLFMA/ref=cm_sw_r_cp_api_glt_fabc_ZJWWH4ZW3JXHXF89RMN3?_encoding=UTF8&amp;psc=1

AWS Import/Export Snowball >Snowball is a petabyte-scale data transport solution that uses secure appliances to transfer large amounts of data into and out of the AWS cloud.

Really great idea! This gets asked about so often.

• Seems like it would be appropriate to add Huginn to "Content Extraction" as well
• You could add n8n to "Open Source" and 'Extraction.
  
• You could add beehive to OpenSource and `Extraction too.

for future reference, this addon/extension will bypass some paywalls, works on firefox/chrome, and you can make requests for large or major sites. next time you'll be able to check if there's something worth paying for

https://github.com/iamadamdev/bypass-paywalls-chrome

I meticulously sort my data into different folder systems so that I know exactly where ever file should be located.
But if you want to be able to search all your data, I would recommend Everything. It indexes all your files, making them instantly searchable, and the even have guides for setting Everything up with a network drive.

1) Install a second drive which is >= 4 TB
2) Download and install Veracrypt
3) Create & mount appropriate Veracrypt container(s) on 2nd drive
4) Copy all files from 1st drive into Veracrypt container(s) on 2nd drive
5) Dismount Veracrypt container(s) on 2nd drive
6) Ensure that you will not forget the password(s) to your Veracrypt container(s)
7) Give your friend the 2nd drive

It can get expensive, but trayless drive cages can help with this.

This looks kind of interesting:

https://www.amazon.com/Loncevon-Portable-Computer-Raspberry-Headphone/dp/B06XQJVXHL/ref=pd_lpo_sbs_107_t_0?_encoding=UTF8&amp;psc=1&amp;refRID=T9344THZZXWVJ8GXJDZE

Per TB? No. Per 8TB stick? Also no.

I'm assuming it's 2 of these: https://www.amazon.com/Sabrent-Rocket-Internal-Performance-SB-RKTQ-8TB/dp/B08957PT2K/ref=sr_1_3?dchild=1&amp;keywords=sabrent+ssd+8tb&amp;qid=1624433681&amp;sr=8-3

Not to rub it in but I think you would have been fine is you hadn’t reformatted at every trouble point. But no point rehashing.

I had a similar setup to you, except I had some 2TB drives mixed in. I have fought with external drive bays for years and finally threw in the towel. I decided to build a new box to act strictly as a NAS + Plex server and cut down to 4 disks.

I went with this case (Mini-ITX NAS Server Case with 4 Hot-Swappable HDD/SSD Trays 200W PSU https://www.amazon.com/dp/B01HIUOLZ4/ref=cm_sw_r_cp_api_OroyAbC0H0G4S), some new RAM, an asrock J4205 and some spare parts. Less than $250 all in.

4 storage disks all part of the Drivepool data pool. SSD OS drive. Plex hardware transcoding. It is running smoothly and drawing less power.

My only problem is what to do with the pile of smaller disks and external drive bays laying around.

Insta has a limit on how many 'views' one can makes a day, some of these apps take that API into account, but if you open Insta on a phone or your PC, then you will exceed that limit and thats what triggers it.

either use/find apps that can have a lower limit on the 'requests' to the website, or simply do not view the website within a 24hr period of ripping.

its explicitly explained at the top of their FAQ here: https://instaloader.github.io/troubleshooting.html

> The hard drives are not hot swappable, because we wish to avoid the operational burden of field service

More likely they just swap entire units out once they're sufficiently degraded.

After seeing so many poor experiences with hard drive shipments from Amazon I was hesitant on upgrading my storage. I had bought two sets of HDDs from Amazon prior (2014 & 2016) and also had great packaging. I entered this community some time last year and one of the things that sticks out is the “HDD packaging sucks” posts. Also, the EasyStore shucking.

I went and bought 4 EasyStores so I could shuck them when the price dropped to $150 last August but I ended up using them as regular externals for back ups, and for family stuff instead.

I had these hard drives sitting in my wishlist for a long time, I got notified about a $29 price decrease ($254 to $225). It still wasn’t as great as an EasyStore but it made me feel better about buying a bare drive. The night I ordered, I had a dream that I had already received the hard drives. One came in retail packaging, the other was just a bare drive in the box, the third was a 2.5” drive with no packaging either. The rest were a mix of cables and adapters.

My eight HDD’s appear to be healthy and I can’t wait to plug them in this evening when I get home from work.

These are 8TB SeaGate IronWolf drives, I was bummed about the deal they offered was only for the Pro, so when I saw the regular IronWolf drop in price I pulled the trigger. Not sure how much longer the sale will last so here’s the link to the drive.

You should check out blue maxima’s flashpoint. It started as an effort to preserve all flash based games before the death of flash, but after they were largely successful in that they’ve been branching out to include a bunch of other web based game types. The latest version has a bit over 100,000 titles included. It’s a free download, but even the zip file is the better part of a terabyte. Here’s their site:

https://bluemaxima.org/flashpoint/

You need to configure Cloudflare & B2 to work together. When setting this up with a private bucket (plenty of docs on doing that on both Cloudflare's docs and Backblaze's), most of the docs are focused on making the bucket public from Cloudflare's side. You need to create a worker that forwards the auth header to B2.

Worker:

addEventListener('fetch', event => { event.respondWith(handleRequest(event.request)) })

const B2_URL = "f000.backblazeb2.com";

/** * Respond to the request * @param {Request} request */ async function handleRequest(request) { let b2Url = new URL(request.url); b2Url.host = B2_URL;

let b2Request = new Request(b2Url, { method: request.method, headers: request.headers }) return fetch(b2Request) }

Then it's just a matter of configuring the download url in RClone to point at your Cloudflare endpoint. --b2-download-url - doc

There are great ideas and suggestions in this post already, but I didn't see anyone cover these points:

1. Resolution: I'd be hesitant scanning much higher than 300dpi: it slows down every scan to do higher resolutions, fills your hard drive faster, and it's highly unlikely that your prints are resolving film grain that small. I've also seen people waste tons of time scanning at <= 72 dpi, because that's what looked OK on their screen, and be surprised that they just didn't capture enough pixels.

2. Capture all the metadata you can: If there's writing on the back of the photo, scan that too. Was it in an envelope with other images? Add that to the metadata. Sometimes it's just a bit of additional context to push an image or album from being irrelevant into something that's part of a compelling narrative.

(Source: I paid for college working at a digital prepress shop, running a room-sized Scitex film scanner and was their on-prem graphic artist. I'm also currently building a digital asset manager, PhotoStructure).

>most money?

Sell individually on Ebay, list them as 2011/2012 MACBOOK HARD DRIVE

>least effort?

Sell them in bulk or give them away or recycle them? Please don't throw the out. They can still be used.

>any cool things I could do with them?

Build a replica model of the Great Pyramids of Giza or you buy an old desktop and build a NAS using nothing but 2.5 inch drives I guess. ICY DOCK hot swap bay

I've always wanted to try building a NAS with only 2.5 inch hard drives. Dumb but funny project. But honestly I would sell them. You can turn quite a profit on top of the money you already make.

Edit: Grammar

Thanks! SearX looks super interesting.

Not sure Swisscows should be on your list, they self-censor - https://swisscows.com/en/about#family-friendly

I've lived in apartments that the internet was self-censored by the good Christian owners - so family unfriendly content was blocked e.g. pornagrphy and information about other religions.

Qwant also states they use Bing and says they follow "right to be forgotten" laws. Companies can request to removal of content via "dereferencing". So I would say Qwant is in the same boat as DuckDuckGo.

people may also find bulk renamer to be useful, not for TV and movies, more of a general tool.

http://www.bulkrenameutility.co.uk/Download.php

it allows mass regex and many other things, useful when you have a TV show that is not named correctly for automatic tools to match

Not to mention their Wireguard service is pretty fast, and supports port forwarding up to 5 ports.

I had port forwarding setup with PIA previously buts it's a PITA to get setup if you don't use their desktop client. You have to run some script in the background to get the port and keep it alive, and it was super flaky once it did work. With Mullvad, they just give you the ports when you request one, and it's yours, associated with your key and persistent. No script or other hoops to jump through.

Yep, amazon is my go to when I need a Linux ISO

https://www.amazon.com/Ubuntu-Release-4-discs-Installation-Reference/dp/B0047B0U1C/ref=sr_1_2?keywords=linux+disc&amp;qid=1556137286&amp;s=gateway&amp;sr=8-2

> Newest Linux Release

> Ubuntu 17.04

> Platform : Windows Vista, Linux, Windows XP, Mac OS X Intel, Windows 7

> Price: $27.95

also on amazon: 100x blank disks for $16

something doesnt add up here...

Sure, but those SSDs MSRP are around $300+ and they came out a year or so ago.

I would expect that a 1TB SSD with a MSRP of $150 will be around half that next year which will be interesting to have a 1TB SSD for around $75.

But sure, you can get an older 1TB SSD for less than $150 now.

https://www.amazon.com/gp/product/B078211KBB

To the OP and everyone in the comment telling the OP to get a new PSU: you know you can just get some high quality 4-to-1 Molex to SATA adapters right? This isn't rocket science...

Given load it'll be a drag right now, you rom dudes are insane, read this if you want to speed things up and get things we're not hosting on site right now.

You joke, but this is one of the files on his drive:

1242B4BDB7499398B2F6FB846143C590_0058_-_Naruto_-_Saikyou_Ninja_Daikesshuu_3_(J)(Trashman).SAV

You can see the full file list here The-Eye.eu and you can download it here The-Eye.eu if opening it in browser crashes it :)

Not exactly what your concerns are but for me I worry about privacy. Some of the basics I would suggest are running an open source router (dd wrt or pfsense) and locking it down. Doing things like turning off upnp and having a secure Wi-Fi password help a lot. If you can run a secured browser even better.

I also encrypt all my backups and run most my web traffic through a VPN.

Check out https://www.privacytools.io if you want more info.

Or you could just stockpile porn for when the world's servers blow up and sell for a profit. :)

Sonarr and Radarr are media automation tools. They can scrub your favorites trackers and NZB indexers to search for your wanted royalty free shows and movies, add them to the downloader of your choice (a torrent downloader or a binary newsreader) and handle the renaming of the resulting file so that you can have a neatly organised media library.

Pulsarr, which is linked in this post, is a browser add-on for adding movies to Radarr or TV Shows to Sonarr while browsing IMDB or TVDB instead of opening the webpage of your Radarr or Sonarr instance and searching+adding the wanted media here.

Better contrast would be a couple MicroSD.. $300 would get you two 512GB

Side note: anyone know someone at Amazon? 256GB for $10? How fake can you get?!

> When the laptop is unplugged, the SSD is slowly draining the battery of the laptop to maintain state. When the battery eventually dies in that laptop, you lose data on the SSD.

[citation needed]

Once shut down (normally) an SSD will have written out any of its internal volatile state to flash. Continued standby power should be irrelevant (not to mention insufficient to really do much of anything).

> You unplug it for a week and your boot drive is bad. Really.

No. This article is about power loss during operation.

SSDs are little computers. They maintain maps in RAM about where data is stored on the flash. Remove power while they're running, and if the drive has insufficient power loss protection, these mapping tables can get lost or corrupt.

You may also interrupt the disk in the middle of an operation, leaving partially written data on flash.

None of this has anything to do with leaving a disk unplugged for a while, so long as you didn't just yank it while your machine was running.

I hardly use the webgui myself since I use Transmission Remote GUI. The main reason is that I hate how massive in size each entry is. Basically it's love the client hate the interface personally and just personally speaking yours doesn't solve it either for me not that the design isn't nice though.

It's actually super easy if you already know how to use rClone. Go to cloud.google.com and enter your billing details (you'll never get charged). Go to the console and create a "Project." Once you create your project, go to the "Compute Engine" and "Create Instance." Under create instance all you need to do is select "Micro" under the machine type and finish creating your instance. After that just run the rClone install script "curl https://rclone.org/install.sh | sudo bash" and you'll be set... From there you just need to run the rClone config commands to get each of your drives set up.

Keeping movies or pornagraphic content online with large/consumer hosting services can be hit or miss. I've seen posts about Amazon removing and banning people from its cloud services even with encrypted uploads.

Have you thought about setting up ownCloud?

I mean, it's a hell of a deal for the price. But I was having drives randomly die on me, and to remove it was just a pain in the ass...or to add new ones.

I got this one instead: https://www.amazon.com/Rosewill-Rackmount-Computer-Pre-Installed-RSV-L4412/dp/B00N9CXGSO/ref=sr_1_1?keywords=12+bay+hot+swap+case&amp;qid=1575296502&amp;sr=8-1

Not everyone is in the same boat as me though. The one OP is sharing is still a hell of a deal - I just couldn't personally go back to a non- hot swap case.

Worked great for me too! --- Link FYI - https://www.amazon.com/gp/product/B006ZFQNT6/ref=oh_aui_detailpage_o01_s00?ie=UTF8&amp;psc=1#customerReviews

Sounds like something I would attempt to do lol

Get a double conversion UPS and I would have him rethink the hdd and maybe go for SSDs. If he can't, at least use Nas rated hdd's for the extra potential shock.

I would also look into something to mitigate the vibrations, perhaps a ton of these? All Rubber Vibration isolation pads https://www.amazon.com/dp/B002IA0WQC/ref=cm_sw_r_cp_apa_i_yXDrCb14SQDM1

Best of luck to your brother haha

You can use Rufus to build a bootable USB with the Windows build of your choice, all the way back to 1507 IIRC. If you don't have an ISO/disk for the version you want, Rufus will download it and build it for you.

https://rufus.ie/

You can scan the health of the drive using our free diagnostic software SeaTools.

If you would like to look into a potential RMA, you can utilize our Warranty Validation Tool as well.

Seagate Technology | Official Forums Team

Prime day lightning deal, $19.47/tb (~21 after tax). If that is an amazon specific model and not the new main design, guess i celebrated too early.

Good point on smr, ill move shares onto it that are mostly only read. Actually i should go thru my array and see if i have other smr drives because i think i had 1 or 2 in the past, and if so will move data around so they are mostly read only data.

https://www.amazon.com/gp/product/B07CQJBSQL/ref=ppx\_yo\_dt\_b\_asin\_title\_o00\_s00?ie=UTF8&amp;th=1

drive - Seagate BarraCuda 3.5 (SMR)

Nanoxia Deep Silence 6 there is so much room inside I was able to stack 2 Sans Digital HD Racks inside giving me a total capacity of 25 drives. I have 17 (10-12tb drives) in there, so room for growth

Galaxy brain move is buying naptha, lye, and a glass turkey baster together so you can extract DMT while watching Doctor Strange. Too bad Amazon doesn't sell Mimosa Hostilis anymore.

Dual 8TB still showing $170 for those that missed the $80 10TB.

&#x200B;

As mentioned elsewhere in this thread, the duals are supposed to be guaranteed reds and shucking is allowed.

&#x200B;

edit: link https://www.amazon.com/gp/product/B074QW86T4/ref=ppx_od_dt_b_asin_title_s00?ie=UTF8&amp;psc=1

SSDs cannot be used for data archive or backup. Period. It should be assumed that unplugging them completely wipes the drive.

Note, there is an even nastier effect that most people don't understand.

When the laptop is unplugged, the SSD is slowly draining the battery of the laptop to maintain state. When the battery eventually dies in that laptop, you lose data on the SSD.

But what about a desktop that doesn't have a battery backup?

You unplug it for a week and your boot drive is bad. Really.

So when you get an SSD for your desktop, you also really need to get a UPS as well.

In the past, I have used an application called Sitesucker. It downloads the entire website and then you can browse it just like a normal online blog but it will be offline, saved, and archived on your system. You can find alternatives to that over here.

> Bonus tip : don't use a password that might listed on https://haveibeenpwned.com/

God damnit, nearly every one of my email addresses are on there including ones I use for banks/brokerages. Now I have to create another email to use with just those services.

Sign up for a Free EC2 instance

When your on it type

sudo apt install unzip

wget http://downloads.rclone.org/rclone-v1.30-linux-amd64.zip

unzip rclone-v1.30-linux-amd64.zip cd rclone-v1.30-linux-amd64

sudo cp rclone /usr/sbin/

sudo chown root:root /usr/sbin/rclone

sudo chmod 755 /usr/sbin/rclone

Then use by

rclone config

Follow on screen instructions to setup for Google Drive

Then

rclone copy $REMOTENAME:$REMOTEPATH ./

unzip $FILE

rm -rf $FILE rclone-v1.30-linux-amd64.zip

rclone copy ./* $REMOTENAME:/Extract/

WinDirStat to the rescue!

On linux, ncdu is basically the command line version of this.

Edit: It's strange that the professional version of TreeSize doesn't work for you, I've never used it but based on the name, TreeSize sounds like it has one goal in life: give you the size of directory trees. Having it not work is a grade A failure (considering how simple it is to determine a folder's size, any coder could code up something that does this in a few minutes).

For my SATA card I use this cable and since I have a couple drives connecting through onboard SATA ports I use a matching cable. I like these because the cables are thin and pliable so it is easy to maneuver them around inside the computer and don't take up a ton of space.

Egg crate foam . Arrange it around the NAS, but not against it, and make sure to leave air vents unobstructed, and air flow around the foam open, you just want to obstruct the sound waves not seal the air off.

Id get somthing like this. https://smile.amazon.com/Synology-bay-DiskStation-DS918-Diskless/dp/B075N1Z9LT

That way you don't have some custom box that the next guy inherits, support for the box with documentation.

Guys, it's possible that you are grabbing the files too quickly and being shut out because of it. Try using the --wait argument with wget. You can add a random time interval or a specific time to wait. You can read more about it here.

You guys know it, Kiwix is an offline reader that allows users to browse entire copies of Wikipedia (ca. 83Gb for the whole thing, incl. images), StackExchange (new release coming up soon), the Gutenberg project, etc. stored locally. The project is fully FOSS, lives from user donations, and the subreddit is r/Kiwix.

The PR has been made last night so consider it fresh off the press. Feel free to test and report issues here. Thanks to the folks who made it possible.

I wrote this up in Powershell real quick: http://pastebin.com/KTXP08J0

I did some googling (Not binging) and they don't have a easy way to access the archive. You would have to do some .Net programming and create a IE object and manipulate IE. I think if you set this script to run daily you would build a nice collection pretty fast.

Also look on line #9 you will see 'idx=1' you can increment this up to 15 to go back that many days before they remove it from this method of accessing it.

This helped alot and provided the JSON, XML, and RSS urls for accessing the data.

https://stackoverflow.com/questions/10639914/is-there-a-way-to-get-bings-photo-of-the-day

Hope this helps.

I've built a ceph cluster with raw 100TB storage and that's generally considered peanuts

>With an average of 2.7 PB, over 25% of all installs are larger than 1 Petabyte

We're renting bunch of different machines but the storage is mostly in these

If you want the data replicated 3 times and some real HA stuff then you'll be at 1kusd/m otherwise about 300usd/m

It'll be cheaper to buy servers in the long run so I'd advise you look at that route if you have the space and can deal with the noise.

Assuming the tags are decently consistent, exiftool might be useful here. Has a bit of a learning curve though.

Edit I've only used it for images, but ran a quick test and it seems to work:

exiftool -r '-TestName</testdir/${Artist;}/${Album;}/${Title}.%le' *.mp3

Outputs:

'02 Do You Feel Loved.mp3' --> '/testdir/U2/Pop/Do You Feel Loved.mp3' '05 Staring At The Sun.mp3' --> '/testdir/U2/Pop/Staring At The Sun.mp3' '06 Last Night On Earth.mp3' --> '/testdir/U2/Pop/Last Night On Earth.mp3' '07 Gone.mp3' --> '/testdir/U2/Pop/Gone.mp3'

I think the bigger issue is that its 2017 and we still haven't really solved the problem of immortalizing data.

Maybe this subreddit is an exception, but for example, things like sci-hub are almost impossible to backup due to missing peers on the torrents. If sci-hub closes down tomorrow, we are at the mercy of the few lucky ones who were able to finish the archive.

It's the world's biggest free scientific database, and besides Russian natives on their forums, I've seen maybe 1 person on reddit that had a full copy.

IPFS is promising but it's not catching on fast enough IMO.

Think about it, amazon drive dropped unlimited, other providers disappear, people moving from cloud to cloud, hard drives fail etc.

Surely many people in this subreddit are prepared for those scenarios, but many people aren't.

If you are referring to the net neutrality thing going on, I don't know what will happen, but in terms of archiving, I feel like we were ill prepared (except for this subreddit of course).

Awesome!

14TB My Book up as well: https://www.amazon.co.uk/WD-Desktop-Password-Protection-Software/dp/B07Y3JXGPL/

If either of them drops to around £220 before they ship my 12TB, I'll swap my order.

Personally I'd opt for something like this

https://www.amazon.com/ORICO-External-Multi-Protection-Carrying-Briefcase/dp/B0714BK6RX/ref=sr_1_3?keywords=orico+hdd+case&amp;qid=1550417644&amp;s=gateway&amp;sr=8-3

It's not cheap, but it's a lot less than damaged drives and lost data.

I listen to audiobooks with the Smart Audiobook Player
Best player for audiobooks I can find.

But I don't use that to look them up ... I just have my books sorted by genre, then author. By hand .. but then I don't have 1000's of them.

I'm pretty sure this book talked about how easy it was to scrape facebook before they locked down their API.

https://www.amazon.com/Mining-Social-Web-Facebook-LinkedIn/dp/1449367615/

A lot of people probably did this. I remember a talk given in my city, the guy had a few thousand people signup to his app and got millions of entries to his graph database

https://maxdemarzi.com/2013/01/28/facebook-graph-search-with-cypher-and-neo4j/

Popular game devs probably got oodles of data. Must have been awesome having a social graph of the US

>I logged into Photos via web browser last night and initially got a prompt about unsupported videos found and even gave me some options, but I accidentally closed out the window and now can't seem to find out how to get that back.

https://photos.google.com/unsupportedvideos

&#x200B;

https://mega.nz/#!WtcGUaAQ!_pRkdIbAwwCkGKLdRz7dBH8ZnZnDzMQA7_TMAqvxVh0

(credit: /u/fizz_zix src: https://www.reddit.com/r/Vaporwave/comments/3odwei/attention_kmart_shoppers_an_archive_of_instore/cvwl6rs)

On the plus side, Flashpoint has this site archived. Unfortunately, it looks like Posemaniacs is broken in the Infinity version (download-on-demand), which means you'd need to download the whole 532GB package if you wanted to use it.

Nevertheless, it should be possible to download the site (with wget or something similar) and run it in an older version of a browser as /u/SeanFrank suggested.

I'm probably going to be downvoted into oblivion for this, but my fileserver is a Synology DS1813+. I don't have the time or inclination to build and maintain a low-power fileserver, and so I just went with Synology. It also runs a basic (internal, not exposed to the outside internet) webserver, as well as mysql. Anything more complex generally runs elsewhere.

Total Commander can do this - and soooo many other things (OOTB and through plugins) https://www.ghisler.com/index.htm

Can’t live without it on any Windows machine and still miss it on my macs.

See all features here: https://www.ghisler.com/featurel.htm

If he's paying 10/month for 2TB he's not using google's archival solutions (nearline and coldline). That's how much it costs for 2TB of Google Drive storage.

Their archival solutions are substantially cheaper.

I've been using BTSync (now called Resilio) to sync files between computers. I don't use FreeNas, but here is a tutorial to set it up.

Too late for me, I was a big dropbox user for years, they dropped support for modern file systems and I dropped them. I started using syncthing to replace the functionality I lost with dropbox.

Another option is using dash which lets you download not just StackExchange sites content for offline use (and is easily searchable, etc) but also programming language documentation, API documentation, etc.

https://www.amazon.com/Sabrent-Rocket-Internal-Performance-SB-ROCKET-1TB/dp/B07LGF54XR

&#x200B;

check out the video at the end of the pictures.