What is Reddit's opinion of
PhotoStructure?
From 3.5 billion Reddit comments
➔ PhotoStructure website
By popularity on Reddit, this Service is:
100 reviews of this app found across Reddit:
Being the author of a self-hosted app, I'm acutely aware of how many people, both technically adept and technically naive, are suspicious or outright hostile to the idea of self hosting.
Perhaps they've never done devops, or have had poor experiences, or are conflating inexperience and fear with loathing: in any event, don't let it bother you. I also wouldn't invest a lot of time trying to convince others. Just keep on keeping on.
There are great ideas and suggestions in this post already, but I didn't see anyone cover these points:
Resolution: I'd be hesitant scanning much higher than 300dpi: it slows down every scan to do higher resolutions, fills your hard drive faster, and it's highly unlikely that your prints are resolving film grain that small. I've also seen people waste tons of time scanning at <= 72 dpi, because that's what looked OK on their screen, and be surprised that they just didn't capture enough pixels.
Capture all the metadata you can: If there's writing on the back of the photo, scan that too. Was it in an envelope with other images? Add that to the metadata. Sometimes it's just a bit of additional context to push an image or album from being irrelevant into something that's part of a compelling narrative.
(Source: I paid for college working at a digital prepress shop, running a room-sized Scitex film scanner and was their on-prem graphic artist. I'm also currently building a digital asset manager, PhotoStructure).
I'm building one that you're welcome to try: https://PhotoStructure.com
I wrote it to be simple enough for my not-so-tech-saavy parents to install and run, (via a "desktop" edition that runs on macOS, Windows, and Linux), but also runs on docker and NAS. Libraries are portable across machines and OSes.
If you're migrating off of Google Photos, I'd try v1.0 (currently in beta): it'll import the JSON sidecars from a Google Takeout which include album and face metadata. Make sure you extract all the archives into a single folder, though: each .zip/.tgz file contains a random subset of albums and sidecars.
>This makes it mathematically impossible
Oof, impossible is a strong word.
And sorry, that's not how ECC works. You can have data and parity bits flip together on media and pass ECC checks.
I concede that this is not likely, but "unlikely" does not equal "impossible." When you're dealing with 15TB drives, that's 15,000,000,000,000 chances for Murphy's Law to win, per drive (well, some fraction of that, as each byte is protected by an ECC block, but assume you write several times, and waves hand towards statistics book there are several other conflating factors, but this small likelihood is still multiplied by a number in the trillions).
I believe his comment that RAM was a corruption vector is a likelier source of data corruption. It's why I make PhotoStructure verify all disk copies with SHA512 (and on my library, this verification check does sometimes fail, and require a retry of the file copy op).
However, if he'd said that ECC accompanied by ZFS or btrfs data scrubbing makes bit rot all but impossible, then I'd agree.
There are at least three apps you can try for Android—links are here: https://photostructure.com/faq/how-do-i-safely-store-files/#how-do-i-back-up-files-on-my-phone
(I use Resilio Sync, but many of my beta users swear by PhotoSync)
Error code correction is built into modern storage media (like HDD and SSD) which should keep you safe from bitrot. Really old drives may have error rates that exceed ECC, though, due to signal logs from magnetic entropy.
If you've got image corruption from what looks to be bitrot, especially with recently-taken photos, I'd suspect a truncation from your file transfer or backup software, a misbehaving router, or bad RAM on one of your devices. If you don't address that first, no fancy filesystem will save you from borked files.
(Source: me doing research for this article)
I’ll second photostructure, especially for the OPs requirements of using the file structure. Early days, but I like the responsiveness and openness of the developer. It may turn people off, but there’s a paid tier. Decide for yourself: https://photostructure.com/about/pricing/
If you're comfortable with the command line, this is easy to do with ExifTool.
https://exiftool.org/forum/index.php?topic=4409.0
is an example, and if you search on his forum there are many other worked examples.
If you'd rather use a UI, you can use PhotoStructure's "automatic organization" feature (which uses ExifTool under the hood).
Know that different cameras and phones can store the captured-at time in a ton of different tags: https://photostructure.com/faq/captured-at/ (btw: ExifTool won't parse datestamps from sibling files or from pathnames: that's exclusive afaik to PhotoStructure).
One last thing: you can run either of these applications on your Synology, but if it's easier for you, just mount your Synology drive on another computer and run the software there. It'll be a bit slower due to network latency and bandwidth, but if you've got a big library, it'll take a long time anyway.
One last, last thing: take a backup of all your stuff before you do anything, and, preferably, have it disconnected. That way, if anything goes sideways, you haven't lost any of your files.
(you might want to fix the autocorrect typo in your title)
There are several apps that do this really well: Resilio Sync (for iOS and Android), SyncThing (for Android), and PhotoSync (for iOS and Android) will automatically back up your phone to your NAS at home.
You install the software on both your phone and your NAS, and then configure your phone to automatically back up to your NAS.
(Excerpt from <https://photostructure.com/faq/how-do-i-safely-store-files/#how-do-i-back-up-files-on-my-phone>)
Synology has a couple (!!) photo apps that they provide. They both have different features: I'd kick the tires with a small test library first to see what they offer. Both are being discontinued with a single replacement photo app in DSM 7 (which is still in beta: don't try it yet unless you have a full system backup).
There are many other photo management options that will run in Docker, if that's not intimidating, including PhotoStructure (I'm the author).
>transfer data between my synology nas and devices on my network
If your router doesn't have upnp enabled, and doesn't expose any forwarded ports, data over your network should only be visible by computers and devices on your network. If you trust that all of those devices are not running badware/malware, you're fine.
>my banking files etc
You should employ layered security by only having sensitive files "encrypted at rest." Use free software like VeraCrypt or something like that to store your sensitive files encrypted. Make sure you use a strong password, that the password is written down and kept someplace safe, and that you have a reasonable backup system in place (your NAS RAID is not a backup!).
I write software that imports Google Takeout sidecars and album metadata, and after having several hundred users send me their problematic files, I've found that it's a total crapshoot for whether the file's metadata is completely intact, partially deleted, or even changed to be incorrect when it comes back in your Takeout archive.
I ended up having to make a crazy long set of heuristics to get the "best" captured-at time, and it seems to work well.
My point is: don't run a simple ExifTool script on your original files without first having a (couple!) backups.
I've personally messed up a bunch of my original photos because I used an app or tool that rewrote or wiped metadata sections (either by design or due to buggy encoders), so I decided to only use sidecars by default (you can change it though) just out of an abundance of caution.
You could put all of these files in a .hidden or .archive directory (like, .../pictures/.archive/YYYY/YYYY-MM-DD/...). I suspect many DAMs running on UN*X will skip over directories that start with a period. Alternatively, you can mark the directory as hidden on Windows or macOS.
There's also a <code>.nomedia</code> convention on Android that seems to match up with what you're asking here: they're for directories containing files that you don't want in your library, but don't want to delete. (I adopted it for PhotoStructure, but I don't know how many other DAMs respect it, compared to skipping over hidden directories).
Image metadata is a mess. Pretty much all software and hardware vendors can't decide on which standard they want to follow, so depending on the camera you use and the software that's touched your photo, keywords might be in EXIF, IPTC, XMP, or 10 other "tag groups". The list of tags I've got here is from examining literally thousands and thousands of example files I've found on the net or provided by beta users of PhotoStructure (I'm the author).
>Hey team,
Howdy! It's just me at this point, though.
>facial recognition functions
Full disclosure: The beta of v1.0.0 of PhotoStructure currently extracts names and faces from tags that Picasa, Lightroom, Google Photos, and other apps have left behind. Native face detection is coming in a future version soon (as it's currently the highest voted-for feature: make sure you set up an account and vote for what you want next!).
>they'll get copied into the PhotoStructure folder system alongside my spouse's and voila, all our photos in the one place. Is that correct?
Yup, that's what it should do!
>do I just point PhotoSync to an "inbox" folder on the NAS and then have PS monitor that folder?
Exactly: that's what my family does.
>How do we view the PS library on our phones?
I personally have PhotoStructure running on a NAS that's always on. The NAS also runs Resilio Sync (to receive my smartphone photos and videos) and Caddy (which is a reverse proxy that handles https certificate management and basic http auth). My router then forwards 443 to my NAS.
>I've read that I can set up PS on Docker, what do I need to do to shift what I've set up from my mac to the NAS? The library's already on the NAS, so I don't know what exactly needs to move.
Set up docker on your NAS, and then point the /ps/library bind mount to the same library directory that your Mac was using.
>I don't see where that "2GB" (or whatever it ends up being) actually is.
See this: https://photostructure.com/faq/library/#where-are-my-librarys-database-settings-and-preview-files
>questions, but they're eluding me rn.
No problem: know that https://forum.photostructure.com has tons of prior answered questions, and the main website does as well. Most pages have come from answering user's questions!
Author of PhotoStructure here: this is the answer for PhotoStructure. You'll be asked this question on the initial settings page (and you can always change your mind and change it later, but it won't affect the files already in your PhotoStructure library.
There really shouldn't be a "wrong" way to configure these two: whatever other image management or editing software you use, PhotoStructure should accommodate it: https://photostructure.com/faq/system-of-record/
(Feel free to post to the forum if you find that's not the case, I'll look into it!)
Yep, disable Automatic library organization in PhotoStructure and set PhotoPrism to Read Only mode.
Author here: I wrestled with this for a while.
"Forever" software licenses are (almost) never forever: they cover a specific version which doesn't get security or maintenance updates, and give perverse incentives to software companies to only do "big bang" new versions (which requires an upgrade fee for your "forever" license).
It's best for both parties to have aligned incentives: if you, as a user, have the leverage of stopping the prospect of future revenue, that gives the software vendor a (strong!) incentive to give you better support (both by maintaining the software and giving reasonable customer support). If the revenue they see from you is "one and done," you're just hoping the software vendor acts in good faith for all the future work you're asking of them.
I wrote more up here: https://photostructure.com/about/v-0-9/#what-about-perpetual-licenses
>Sharing of directories/images privately. This appears to be on the list, maybe.
Albums and directories will be implemented as hierarchical tags, and sharing will be via a token applied to a tag (so you can easily share a full directory, recursively, for example).
I'm thinking that sharing a single asset will still be via a tag (that's created automatically), just so all "shared" items only have one place to look.
I'd thought of putting share tokens on both tags and assets, but it adds complexity that the user is exposed to when there are several ways to acquire share tokens.
If you're interested in more details, send me an email and I'll link you to the design doc I've got.
I've had a number of my users ask me this exact question, so I wrote up this primer: how do I safely store my files?
If this is a business, keep at least one robust copy, preferably in a NAS that offers data integrity, at home, and consider cloud storage as another backup copy.
"Bad sectors" isn't a case of "the drive may fail." The drive is actively failing.
The drive ships with spare sectors it doles out as it finds defective media. When those run out, you get those errors. Backup what you can, while you can. You currently have justification to buy new drives.
Read this section if the budgetary committee needs more convincing, and the two drives were bought together: https://photostructure.com/faq/raid-is-not-a-backup/#4-data-loss-from-correlated-failures
After several years helping support several different popular photo album packages, and trying many others, I've come to the conclusion that it's:
Trivial to come up with a photo gallery script than handles 10s-100s of photos. These projects spring up continuously, because it's a fun weekend project.
Uninteresting/too grinding to make something more robust that handles semantic duplicates, scales to millions of files, and works cross-platform.
So I quit my job and built it. https://photostructure.com/about/introducing-photostructure/
From the signups and interactions I've seen over the past year, I suspect there will be substantial votes for "only free," but certainly some for "sure, I'll pay a subscription," and a shattering of "no way would I self host".
PhotoStructure will do this (I'm the author): install, sign up for a free trial (if you need more than 15 days, just ask), point your "library" to be the directory that you want to organize into, turn on "automatic organization" and include all the random folders as paths to sync. When the progress bar closes, you should find all your files neatly organized.
Note that PhotoStructure won't move the files that you import. It only copies (just to avoid confusion/calamity if something doesn't behave as you expect). https://photostructure.com/faq/why-doesnt-photostructure-move-original-files/
More information about automatic organization is here: https://photostructure.com/getting-started/automatic-library-organization/
If you decide to use ExifTool, please make backups first. It's really easy to get the command line arguments wrong! (ExifTool is great, by the way: PhotoStructure uses it behind the scenes!)
Howdy! I want this too!
I'm currently using Caddy to share my PhotoStructure library with my family, configured to both do https and basic http auth. There are a ton of alternatives that are pretty easy to set up, too;
https://photostructure.com/faq/remote-access/#reverse-proxies
I will be building authentication/authorization in a future version soon: it's the fifth-most-voted-for feature.
https://forum.photostructure.com/t/secure-sharing-of-albums-and-assets/112
I totally sympathize with your concern, but know that all modern HDD storage does have built in error code correcting, which does detect and recover from small bitflips and surface defects.
The best way to keep your stuff safe, though, is with backups. Separate, automated copies of your data.
Fancy filesystems that support data scrubbing isn't really available on consumer Windows except via the use of a separate NAS. It certainly bolsters and complements built-in ECC, but isn't a substitute for backups.
I did some research about this and wrote it up here.
Yup!
https://photostructure.com/faq/library/#importing-from-different-computer
As you play with PhotoStructure, if you find anything odd or confusing or buggy, please don't hesitate to tell me via /r/photostructure or via https://forum.photostructure.com/. Also, as more-glaring missing features are apparent, post those to the forum so they can get upvoted: it really helps me prioritize what I build out next.
Howdy, welcome to PhotoStructure!
Try ./start.sh --expose to bind to all network ports. It only binds to localhost by default.
More details here:
Nice functionalities! Maybe tangential, but wish there was a way to AirPlay or Cast a slideshow generated via such a system.
The clipping and exporting seem like a neat way to generate a custom video for an event etc.
Have you seen PhotoStructure? It seems like what you’re talking about could build on top of the structuring database used by that system.
I use Nextcloud and point it to an “External” share on my NAS. I then share that out to family so they can upload photos. This folder is also the one Plex looks at. Then I have another folder on my NAS where I use PhotoSync iOS app to automatically upload photos and videos. Something I have been looking into though is. PhotoStructure which may have the AI piece you’re looking for. https://photostructure.com/
It's here (and my wife and family want it, so it will happen):
> 🤖 Add ML-powered face-and-thing tagging. As always with PhotoStructure, all computation happens locally on your computer. Your data stays yours.
It was added in the v0.9.1-beta.1 (v0.9.0-* alpha and beta builds won't be updated again, as v0.9.0 was released).
You should be able to pull the :beta tag and get v0.9.1-beta.3.
Please tell me if the correct image doesn't pull down for you.
This is exactly why I wrote r/PhotoStructure. My photos were all over the place (and some had bitrot, and tons were duplicate, or had been down-sampled and stripped of metadata, ...).
It's free during the beta in exchange for your feedback. You don't have to use it after it gets everything organized for you. You can customize the directory structure, too.
When you install it, you'll only have 3 questions to answer:
You can either pick "auto" scanning (and then it'll look in all mounted volumes for photos and videos), or specify the directories you want to read.
Select "yes" to automatic organization.
Make your "PhotoStructure library" be the directory you want all your photos and videos to be copied into (presumably a directory on your NAS).
Read the (short!) EULA, click start, and it'll give you an ETA as soon as it can. Depending on your computer, network, and disk speed, it may take several hours to a day to compete a library that big. If you can directly connect to ethernet rather than fighting WiFi, that'd help.
Once it's done, if you don't want to continue using PhotoStructure, just delete the .photostructure directory in the base directory of your library. (This contains all the preview images and library database).
/edit: I see you wanted face tagging. That isn't in the current release, but it's on the roadmap.
QNAP will have better hardware specs for the same amount of money than Synology. Both UIs work.
Before you buy, though, search google and r/Synology and r/QNAP for security issues, with an eye towards how each company handles both disclosure and recovery from malware. They are markedly different.
In any event, when you're setting it up, you might want to go through this checklist to make sure you're getting the most out of your NAS.
When I did a bunch of research for this article, I came to the conclusion that M-discs weren't really viable if you're trying to backup several TB. It'd just take too many disks!
If all your files fit on a handful of M-discs, though, it seems like it's be perfect.
I'd also make a second copy using a different disc manufacturer, and then verify all the copies are bit-accurate with something like rsync -c.
Also consider using SeqBox, which allows the disk to suffer from some amount of bitrot and still be recoverable.
I've had several beta users tell me that they lost a bunch on photos using other photo management software and then "clicked the wrong button." It's tough to hear when backups are so cheap these days.
It's also why I decided to never move original files: I only copy them.
I think headlines from several newspapers might be interesting, but 2020 was such an unprecedented 💩🍿, it almost certainly won't be pleasant to reminisce about.
I've spent the last couple years working on a tool to manage photos and videos, and I took a lot of photos (for the time) when my kids were tiny.
I've got many thousands of photos, but I really regret not taking more video and more audio. Even when they're screaming meemies. Capture everything: when they're eating. Or snoring. Or exploring.
The time goes by quick, and those memories are precious.
I'd make an album of them, not global horrors.
I'd also buy some alcohol that can be layed down for a couple decades to mature and enjoy when they're actually people and not larvae. I commented about what to buy in another thread.
It depends on what you want to do.
I've got an old Synology 416j. It's fine for file storage, but it
- doesn't support btrfs, so there's no data integrity
- doesn't run any of Synology's current batch of backup software without crashing (I had to run rsync via cron, set up via the terminal)
- come to think of it, I haven't had luck running any of Synology's apps on it, really, other than DSM.
If you're spending hundreds of dollars on the drives you're putting into the device, I'd argue that it's worth spending a bit more to get a device that at least supports btrfs.
With only several thousand photos and videos, that'll fit on a $10 USB thumb drive. Just realize those are only good for ~5-10 years (depending on storage conditions and the chips used in the drive). It's not "forever" storage.
If you want a bit more safety, you can put them on a couple external drives.
I had a bunch of my beta users ask how to safely store their photo and video library, so I wrote this up.
The issue was twofold: PhotoStructure automatically pauses sync if there isn't "enough" free disk space (to prevent the system from crashing), but there's a bug in v0.8.3 (that I've addressed in the next release that caused the progress panel to not always show up.
This progress panel says "Sync is paused: insufficient disk space on /", which would have been helpful... Sorry about this bug!
The minDiskFreeGb setting defaults to 6GB. For now, you can override this by setting the environment variable PS_MIN_DISK_FREE_GB=4 (but you'll want to give it more disk at some point soon).
I kept getting asked this question by my beta users so I wrote this up: https://photostructure.com/faq/how-do-i-safely-store-files/
An external drive instead of a NAS is certainly a cheaper option, but make sure you have at least 2 copies of everything. More details are in the article.
Hit the "down" arrow in the bottom middle of the screen, or hit the s key: that shows the "streams" that asset is associated to.
The next version changes this icon to hopefully something a bit more intuitive.
See also:
I completely agree that software and humans are fallible, but redundancy is also a thing.
Are you aware of any reports of AWS, GDrive, or BackBlaze having caused bitrot in customer files? I haven't, but I'd be interested to know.
I think it's far more common to see file corruption due to bad media (especially old optical media), failed transfers, buggy/truncated backups, or buggy/misused software.
(I'm not a "bitrot denier," fwiw: heck, I specifically tell my customers how to avoid it.)
I had so many of my beta users ask this exact thing that I wrote this: https://photostructure.com/faq/how-do-i-safely-store-files/ (if you read anything that doesn't make sense, is confusing, or you know to be wrong, please tell me!)
One thing to keep in mind (that I wish I'd understood before losing stuff): Make sure you don't confuse RAID redundancy with having a separate backup. RAIDs seem to fail cataclysmically as frequently as they gracefully degrade due to controller/PSU/firmware issues.
If you delete all the file variations for an asset from the filesystem, and click "Re-sync this asset", the asset will go away and you'll be redirected to the home page.
In the next release, deleting is handled properly: https://photostructure.com/faq/archive-remove-delete/
Howdy!
Shut down PhotoStructure for Desktops, if it's running still.
Delete the system settings.toml file: https://photostructure.com/getting-started/advanced-settings/#system-settings
Restart PhotoStructure, and it will think it's starting on a new computer. On the welcome page, pick your existing library directory, and it should open up.
If you're using a network fileshare for your library directory, make sure you don't have your library open simultaneously on different computers: that can cause library data loss.
Cheers!
let me just get trough with the documentation and I will be back :).
thank you for the detailed feedback!
>PhotoStructure's UI is mobile-friendly HTML, but you need to set up remote access. https://photostructure.com/faq/remote-access/
I have a reverse proxy deployed for that and will be back with further questions if I need clarifications.
I recommend people use one of several one-trick-pony sync apps, like SyncThing, Resilio Sync, or Photo Sync.
https://photostructure.com/faq/how-do-i-safely-store-files/#how-do-i-back-up-files-on-my-phone
Howdy! I'm the author of PhotoStructure, which handles image and video deduplication server-side: I've spent a lot of time with this issue, and you might want to read this article to know what sorts of dupes you want to handle.
Whatever software you end up using, it will need to keep a database (or something like it) that stores a "hash" of every image that you're try to deduplicate, or performance will be terrible.
Hash-alone strategies have high false-negative (missing actual dupes) and false-positive (where different photos are erroneously gathered together) rates: it's the nature of using a hash to have collisions. I had to teach PhotoStructure to use image metadata heuristics as well to get good accuracy. (BTW, if you use PhotoStructure, and see any issues, tell me on the forum or on discord, and I'll fix it in the next release!)
Got it.
Know that there are a couple bugs in sync scheduling that I've found and fixed from v1.1, and hope to ship soon in the next release. Details are here:
> I do wonder why I the copied files are 15Gb where the originals were way over 150Gb. Are the photos being compressed or is this just a reference for the original photos?
PhotoStructure will copy the original photos and videos (and sidecars) into your library directory hierarchy, but it also creates a bunch of web-sized preview images (and transcoded videos) for the UI (that you can safely delete later if you end up not using PhotoStructure). Those previews are in a hidden .photostructure directory that you'll find in your library directory: <https://photostructure.com/faq/library/#where-are-my-librarys-database-settings-and-preview-files>
FWIW, if you didn't have an active PLUS plan, PhotoStructure won't have copied the files into your library (as this is a PLUS-only feature). You can tell what plan is active via the About page. I suspect you're on LITE, and the only files taking up disk space are the previews.
> when I try to look at the photos in the app it shows way less than in the folders
PhotoStructure has a bunch of filters to prevent low resolution and corrupted images from being imported into your library. There's a bunch of details here: <https://photostructure.com/faq/why-is-my-file-missing/>, and I'm working on a feature right now to make this issue less enigmatic.
PhotoStructure should work for this use case!
First get it installed (it runs on your desktop desktop or your server
By default, PhotoStructure will organize (copy, not move) your photos and videos into
YYYY/YYYY-MM-DDsubdirectories, but this can be configured (if you want a different pattern, ask and I'll tell you how to configure this).Sign up for a free PLUS trial. You'll have 2 weeks to cancel, which is plenty of time to get the first import done (and you'll get a reminder warning email before your trial ends... I don't like surprises, and I assume my users don't either...)
On the welcome settings page, know that your "library" directory will be where your newly-organized files will all be copied into, so make sure it has enough free disk space.
In the "Where else are your photos and videos?" section, either leave it on automatic, or point it at your old hard drive(s).
In the "May PhotoStructure organize your photos and videos?" section, say yes.
If anything goes sideways, ping me.
I'm the (solo) developer: feel free to ask me anything, either here, on /r/PhotoStructure, on the forum, or on discord.
(You're correct about being new, though: there are a ton of features I'm looking forward to adding, but the libraries are uniquely cross-platform ((and cross-volume)), the deduplication is quite solid, and the UI is snappy even when served over slower bandwidth connections).
For photos specifically, I plan to give PhotoStructure a try as soon as I get some time. Looks like it has some nice features and also has a heavy emphasis on keeping your data in your own control, self hosting, etc. It's a relatively new piece of software I think but I have high hopes for it moving forward.
Howdy! Your situation sounds just like mine, which is why I'm writing PhotoStructure: if you've got any questions, I'm here, and there are tons of details on the website: https://photostructure.com/faq/why-photostructure
Cheers!
Howdy: author of PhotoStructure here: I've soaked in this problem a bunch, and feel your frustration of not wanting to waste disk space on JPGs.
HEIC images aren't natively supported on a bunch of different browsers (including, ironically, mobile Safari on prior iOS releases). Some HEIC images will include fullsize thumbnails in their EXIF metadata, which you can extract quickly-ish (under a second), but the embedded orientation may be incorrect, so your image organizer needs to add additional heuristics to handle that properly.
Also, (more irony), HEIC images from newer iPhones are in a different format than older devices, and macOS's own image tool, sips, doesn't handle the different formats correctly, which requires additional heuristics.
Your concern about image quality when converting between HEIC and JPEG is warranted, but I've found that as long as you use a reasonable converter (like sips or libheif), a high-quality JFIF encoder (like libvips), a library that retains the color gamut profile (for HDR), and a high quality JPEG setting (like 85% or higher), the image quality is indiscernible (PhotoStructure does all these things, and each of these aspects were non-trivial to discover).
Short story long: whatever image organizer you go with, it won't be performant for browsing unless all these shenanigans are done at import time. I'd love to do on-the-fly rendering, but I've found that even a 1-2 second lag between switching between images makes browsing a chore (which is why my goal is to render every page within 100ms, and PhotoStructure easily manages that, especially on hybrid libraries).
There are several great options for getting your files synced from your phone to your NAS at home: Synology and QNAP both have mobile apps, and there are these as well:
https://photostructure.com/faq/how-do-i-safely-store-files/#how-do-i-back-up-files-on-my-phone
(My family uses Resilio Sync on our iOS and Android devices, fwiw)
Look into PhotoStructure. It may not do exactly what you want but the dev is very active and adding features all the time. Interestingly enough I talked with him about a feature similar to what you are describing and he was not opposed to the idea. However due to the nature of the topic our convos were offline and I can't share any more details than that.
Also he was open to the idea of plugins or modules that the community could develop so id look into it if I were you.
PhotoStructure currently imports tags from Google Takeouts, and ML-based search and tagging is coming soon.
I agree with your sentiment: most for-profit software will try to introduce some sort of "friction" that makes it a PITA to switch to a competitor.
However, PhotoStructure is not most for-profit software: I don't have any investors pushing for those sorts of tactics: it's just me doing everything.
I believe well written software (closed AND open source) should:
Leave you in a better place than you started if the software goes away
Should adopt standards whenever possible and avoid proprietary data stores
I bend over backward to not be the "authoritative store" for any action you do in PhotoStructure: everything is stored in sidecars (or the original files: it's configurable).
Metadata is also stored in a clearly designed and commented SQLite database, which is used to make the UI and syncs performant, but it's treated as a possibly-out-of-data data mirror, not the authoritative store.
Author of PhotoStructure here: know that you can adjust how much space previews take up. See <https://photostructure.com/getting-started/how-much-disk-space-do-i-need-for-my-photostructure-library/#how-can-i-minimize-the-disk-space-that-photostructure-uses>.
PhotoStructure doesn't do any ML auto-categorization yet, but I'm working on it, and it will be in a release soon. The feature to vote on (and discuss) is here.
I'm assuming you want multi-year options to get a bigger discount than the 30% annual discount from monthly subscriptions?
If that's the case, know that I'm <u>liberal</u> with additional discounts. If you're a student, or medical professional impacted by COVID, you get 50% off. If you've an open source developer or maintainer, I'll give you between 50% and 100% off (depending on if PhotoStructure uses your code). If you've given substantive feedback, or helped debug issues, I'll give you a discount.
Basically, just give me an excuse, and I'll give you a discount coupon.
Feel free to try out PhotoStructure: I wrote it to help me dig out from my digital mess of dupes and disorganization.
You'll need to set up a free trial to enable "automatic organization." Ping me if you need a longer trial or have any questions.
Yes, and the directory format is customizable as well. Details are here:
https://photostructure.com/getting-started/automatic-library-organization/
My network is spotty this weekend, but ask me anything either here or on the discord or the forum and I'll get it sorted as soon as I can.
Sorry that it's taking too many resources on your NAS!
I've done several things to make sure PhotoStructure normally runs without affecting system responsiveness, so something in the following is amiss:
All the processes are
renicedto run under only low priority. Do you see this happening properly in your process table?PhotoStructure measures your system and schedules what it hopes is a reasonable number of concurrent
sync-filejobs. You can check this on the about page. Are these numbers seemingly too high?The default scheduling should keep the system at about 75% utilization during imports. You can set the
cpuLoadPercentto something like 50 or even 1, which will only schedule one file to be imported at any given time.
https://photostructure.com/getting-started/advanced-settings/
I tried it. After 48 hours of it indexing my photos folder it was still munching through 2009. Far too slow for me. It'd take about a month and about 700Gb of space to get up to date with my photo collection. I otherwise really liked it.
Next on my list is Photo Structure as thus should be a nice easy install and quick index. Fingers crossed
Backup everything first on to a separate drive (or drives) that you can disconnect from your NAS. When you've got a full copy (or a couple) of whatever you care about, "messing everything up" just means you factory reset and start again.
It makes everything a lot less stressful.
To get your phones backed up, there are several solutions: synology has a backup app, but there are two other apps that work well: Resilio Sync and PhotoSync. Having alternatives is never a bad thing.
You also asked what model: you may want one that supports btrfs and docker.
Many people seem to like Moments, but it didn't work for me. I'm building [an alternative](/r/PhotoStructure).
Thanks for sharing.
I've also been bit by subscriptions, and feel the same way about my photo library--it's what drove me to design how I set up PhotoStructure's free and paid tiers:
TLDR: if your subscription lapses, you still have access to your files and your library.
In your last sentence, you said
> I'll happily buy the software and pay an annual maintenance fee for upgrades and support (assuming its good value) - but I won't pay a subscription...
The software industry has happily muddied the waters when it comes to these terms. Are you meaning you'd pay for security and bugfix updates, but not feature updates?
As PhotoStructure is (currently) a one-person show, I can only support one version: the currently released one. New features and bugfixes and security updates all get rolled into the build. Pre-releases ensure each stable release is strictly better than the previous stable release.
You can check out my release notes and the forum to get a feeling for how transparent I am with development and support.
I started trying to organize my photos and videos into YYYY/YYYY-MM-DD or YYYY/YYYY-MM-${EVENT_NAME} subdirectories, and then add symlinks to those files that lived in other directory hierarchies, but wasn't able to stay on top of it.
I actually wrote software to do robust captured-at extraction, but ExifTool will get you 80% of the way there (including moving your photos and videos into these subdirectories). You may want to do deduping as well, and depending on how bad your current situation is (mine was bad), just de-duping files based on SHA may be "enough."
The only thing I'd caution against is a directory structure that results in more than several thousand files in a given directory. Most OSes won't outright crash from gigantic folders, but if you ever export that directory onto spinning rust or via a remote file share, it's certainly not fun to wait a while navigating through and between those directories.
My first de-duplication heuristics only looked at resolution and last-modified time, but I've since adjusted my heuristics to use a rough approximation of resolution (so small crops aren't penalized too much).
Even so, some beta users wanted to tweak those heuristics, so it's now a setting in v1.0.0.
Thanks for taking the time to share!
(Sorry for the inexact quotes, the iOS Reddit client doesn't seem to support reply quotes)
> docker-documentation
I don't know if you saw the docker-compose instructions, but it's an annotated file as you suggested.
> ENOENT
I'll try to reproduce this. Thanks for the heads-up.
> ... Library is current empty ...
This should be fixed in beta.4
Users, sharing, titles, and descriptions are on my to-do list.
> [DNG and JPG with the same name]
These won't be grouped together unless PhotoStructure has determined that they have the same captured-at time. Smartphones and cameras happily create colliding-basename images all the time.
> [Navigation woes]
How would you improve this? Just skip over the intermediate folders?
Cheers!
>added the ability to create a simple album
I don't know if you tried the stable version or the v1.0-beta, but v1.0 added automatic albums based on folder names, so depending on how you've got your images organized on your disk already, that might help.
>I have scanned thousand and thousands of old negatives from family collections.
Ah, I do too: it's why PhotoStructure's captured-at extraction code is so elaborate.
If you found that the date was incorrect, please send me the full path to the file, and (as long as there's nothing private in the image), the original file, and I can take a look.
Hi there, author of PhotoStructure here!
OP, the main concerns I'd try to address with (any!) image management solution is having an "escape route" when (not if) the company goes away.
Are my files left as I want them, or are they stored in some proprietary folder structure?
Did they store all the changes I made while using the product as standard file metadata, or is it only in a proprietary database?
As my library grows, how painful/inconvenient will it be to upgrade storage? Is it easy to make backups?
(I've personally been burned enough by prior image organizers to make these be central design tenants of PhotoStructure).
Ouch, maybe you'll have to de-duplicate the files on your own with a separate app. for example I'd use gemini from macpaw (comes with a setapp subscription) to do it on my mac.
I know photostructure manages duplicate photos, and has settings in place to customize it https://photostructure.com/faq/what-do-you-mean-by-dedupe/
No great answers for this, sorry. Browsing photos on a NAS is an open question for years.
I run PhotoStructure on my NAS in Docker, which gives me a local webpage that I can use to see random photos or to drill down into specific days/times. It just got HEIC/HEIF support, and they are working on Live Photo support as well.
Alternatively, you could import the photos into the Photos.app in macOS, but not move the photos into the library (just “referencing” the files on the NAS). This only works on your laptop of course, so not great.
While Googling around, I found Mylio - which seems like it might fit the bill. Not used it and don’t know much about it.. but super impressive set of features.
SQLite shouldn't require any external installation step: it's a library that gets compiled into the application itself (PhotoStructure uses it and it's great).
The only thing you have to be aware of is that it won't work on a remotely-mounted filesystem: it needs to be a local volume.
What do you mean by "change the date"?
If you mean the ctime [1] or mtime, many/most won't retain this sort of stat metadata.
If you mean the captured-at time stored in metadata, if the service stores your file as uploaded, the tags will be retained (because this information is actually encoded within the file itself).
Google Photos in "high quality" mode used to strip most of these tags, but seems to retain them now (unless you download your file via their API).
[1] <https://en.m.wikipedia.org/wiki/Stat_(system_call)#ctime>
I have considered using plex for photos but I keep hesitating due to unfavorable comments and experiences i keep seeing. I have considered using photoprism. I have also looked at photostructure. Anyone have any experience with photostructure? https://photostructure.com/
Howdy! Author of PhotoStructure here.
You can run PhotoStructure on a pi: it's just slow at importing. My pi is in my CI build cluster for PhotoStructure, and it takes > 4x longer to complete tests than my > 10 year old macbook.
If you're patient, have a smaller library, or run your initial import on a beefier box, it'll do fine:
https://photostructure.com/server/photostructure-for-node/#rpi-install
Howdy! I'm the author of PhotoStructure.
>I don't like the idea of it being automatic
I'm an old unix greybeard: I'm building a tool that I want to use, and my less-tech-literate Mom and Dad can use comfortably as well.
Know that you can do everything either via the UI, or via the terminal, if you want ultimate control of what it's doing. More details about command-line control is here: https://photostructure.com/server/tools/
And a glimpse of the level of configurability available is here: https://github.com/photostructure/photostructure-for-servers/blob/main/defaults.env
You could try PhotoStructure: it'll find dupes (and organize) your photos and videos. it's free during the beta in exchange for your feedback, but the beta is ending soon. Lots more details on the website and forum.
It won't delete duplicates (because I'm chicken), but it'll do all the heavy lifting work so you can easily deal with dupes as you wish (via the terminal).
I'm the author, btw.
Yep, I'm running it on a Docker on the NAS. They have nicely detailed instructions on the website, and the developer is super-responsive. I browse the library via a locally hosted NAS page as well.
While I understand the turn-off in subscription payment for storage (when it was once free), think of it as paying for the excellent feature set that is Google Photos. Can they really be beat for the quality and reliability of their service? The closest is iCloud (if you're on iOS), and even in that situation I simply use both.
To be honest, I do not see any way in which a third party (outside of Google and Apple) can make a service as reliable and full-featured. Even less so, something that can run effectively on a storage box. I recently discovered PhotoStructure to browse my backup store of photos on the NAS, but Google/iCloud Photos are too well integrated and actively developed to be replaced, per se.
The node or docker versions let you run individual tools, but the "If sync is still stuck for you" instructions I posted will work with any edition running on Linux.
You're on docker, right?
Default settings options should scan all bind mounts automatically, but you can either explicitly add the directory to your "scan paths" or you can run it manually with docker exec -it <container name> photostructure sync --verbose <path to bind mount within container that you want to sync>
(you'll want to have gone through the settings page once, though, to make sure the library is all set up).
If you find that you're fighting docker, you can also run it "native" on your fedora box: https://photostructure.com/server/photostructure-for-node/
If you prefer the terminal, you can actually run a sync by hand, too:
https://photostructure.com/server/tools/#manually-importing-files-and-directories
if you run the command with --verbose, you'll see what it's doing (but note: --verbose is verbose).
The home page, <http://localhost:1787/tags> (season "localhost" to taste) should have a progress bar at the top, like
https://photostructure.com/img/2020/06/rebuild.jpg
if it doesn't, that's a bug: please tell me what browser and extensions you've got going on.
If your Fedora VM is on a beefier box, then certainly run the initial import there.
One of PhotoStructure's unique features is that your library is both cross-platform and cross-machine, so you can move it around and not have to re-do all the prior work.
Details are here: https://photostructure.com/faq/library/#importing-from-different-computers
Which Synology NAS are you using /u/waltonnerd? The docs say 2GB RAM is the recommended minimum, I was wondering how you find the performance with the NAS that you set it up on?
OP: sorry for your loss. One of the reasons why I have so many dupes is because I inherited all my Dad's drives when he passed, so I can relate.
If you hear sad clicking noises, you may be able to still mount the drive by shucking it out of it's external housing and attaching it either as an internal drive, or by using a drive dock: a couple of my ancient external drives made sad clicks of doom, but spun up happily when shucked (possibly due to better power output from a better PSU?)
I'd buy a 10+TB drive and use PhotoStructure to get everything into a single, deduped directory hierarchy. Just attach all the drives, use "automatic scanning" and "automatic organization," and after a while (depending on how fast your computer is and how large the library is), you'll have everything copied onto your new drive into nice YYYY-MM-DD folders (with a web UI to boot).
The currently-available beta version is free in exchange for your feedback. I'm hoping to conclude the beta program by the end of this year, though. Tons of details on the website, forum, and [subreddit](/r/PhotoStructure). (I'm the author, btw).
If you just need an app to copy your photos from your smartphone to your NAS, there are many to choose from.
If you were referring to backing up you NAS, the page I linked to also has some recommendations for that as well.
(I'm the author of PhotoStructure, btw)
You'll get this info emailed to you if you subscribe from the home page, but if you'd rather skip that, here's the link to the desktop installers, and this discusses the difference between server and desktop editions.
OP: you probably want some sort of multiuser support to share subsets of your library. This is coming soon. Until then, many of my beta users are putting a reverse proxy in front of PhotoStructure with basic http auth.
SyncThing and Resilio Sync work on android. Resilio for iOS. https://photostructure.com/faq/how-do-i-safely-store-files/#how-do-i-back-up-files-on-my-phone
I really like the one-trick-pony apps like these: if they give you attitude, you can switch them out fairly painlessly.
>Is the software open-source or proprietary?
I've open-sourced several libraries that were born from PhotoStructure (see the GitHub repo), but the application itself is closed-source. More details are here.
What's an "executable front end"? An application?
Unfortunately I haven't shipped face tagging yet. It's on my todo list. Feel free to try out what I've released, though: the beta is free in exchange for user feedback.
So you need:
Folder-based "albums"
Do you need browse by location? (Do the images have gps tags?)
Do you need to create albums per client using several location samples, or are you doing ad-hoc searches (and what do you want to search by? Location name?)
Have you annotated the images (with title, description, or caption? If so, what fields did you use/ what software did you use to add those annotations?)
(I'm working on a DAM that will have 1) and 4) in the next release. It's always better to build out features with specific use cases in mind).
Sorry, this thread fell off my radar and I didn't back to you.
Know that the RPi will complete an import of any size library, it'll just be glacial.
If it only has to deal with new files, even if it's glacial (like, 10 seconds per image), it happens in the background, and will get done when it gets done.
What I'd do in your situation is "initialize" the library on a beefier box.
Make sure you've read about volumes, to make sure the import work from your beefy box is usable by your RPi. Especially read this caveat about pathnames.
Then, once this initial import is complete, copy the library to your RPi, and then let the RPi be the "authoritative" library from then on. No need to do any preview or library syncing: just copy new files to your RPi, and PhotoStructure sync will find and import those new files.
> no keywords in the keywords tab
Well, dang!
PhotoStructure looks in a bunch of different tags for keywords.
What software did you use previously to add keywords to your photos and videos?
If you used Picasa or digiKam (and many other image organizers): make sure you set the preference to export those keywords back into the original files, or into sidecars.
If you're comfortable with the terminal, you can run ExifTool against the files in question to see what tags are present (like exiftool path/to/image.jpg).
PhotoStructure only knows to look for keywords in the file itself or in sidecars: I haven't built out "import from Apple Photos library database" functionality (yet).
> Should I wait or rebuild or something
Running rebuild or sync shouldn't change what's already in your library, unless you change settings (like if you add a drive to scan, or change other library settings).
Howdy! I'm the author of PhotoStructure. Know that everything runs on your device: there's no cloud component, and your photos and videos don't get uploaded anywhere. Your stuff stays yours, even if you cancel your subscription.
The beta is free in exchange for your feedback, and I'm giving discounts to my beta users that provide feedback, but know that the current version is the last beta.
I've wrestled internally a lot about pricing, and wrote up a bunch of details here. There's also a discussion over on r/PhotoStructure.
Windows has slow I/O (compared to the same hardware running Linux or macOS*), but for just copying files, it should be fine.
I suspect you're either dealing with a failing drive, or you're fighting your antivirus app. I'd just use the built-in Microsoft Windows Defender, which seems to not interfere too much with simple file copies.
Open up the task manager while a copy is going on, and sort by CPU used. That process is the culprit.
- I never really grokked how bad it was until I had side-by-side tests that run on Windows and Linux VMs on the same host. A cross-platform project of mine has several thousand tests, many are I/O bound, and Windows is always the last to finish.