Shotwell can handle photos and tagging, but not sure about something that can handle the "whole package" though. Linux applications in general tends to be more "Do one thing good" than be one big suite of software tools.
Just some thoughts about Shotwell. Using Shotwell to manage a few thousand photos on a laptop has worked pretty well for me, but considering the use case here with two orders of magnitude more data, I just wanted to bring up a few points.
From my limited experience, Shotwell needs two things during importing, CPU and disk IO. It hashes every photo into a "photo ID" that goes into its database. The hashes are used for de-duping and more. Shotwell also monitors folders for changes, however this can (and probably should) be disabled during initial importing (see https://wiki.gnome.org/Apps/Shotwell/FAQ).
Shotwell uses SQLite which, although it is a great little database, doesn't scale well. My advice would be to time the importing of a month's worth of photos (or however they are stored) and then compare the time of importing another month's worth, and another... then graph it to get a rough idea of how much time the full import is going to take.
Another advice would be to start up a Task Manager before starting any import, Shotwell's own progress indicator can be unreliable, sometimes making it difficult to tell whether importing has stalled or not. Although Shotwell crashed or hung a few times during my own initial import, it never messed up or lost any data.
Another idea that might be worth considering is to do the importing on another machine with beefier CPU and IO, if the MacBook itself is too weak. How to transfer the data and database is also described in the FAQ.
The latest version is 0.22.0, not sure if it's available for Elementary. In any case, you'd probably want a version > 0.20.0 due to a bug in earlier versions (see FAQ).
An interesting (and slightly scary) endeavor in any case. Good luck!
I've never tried this...but it's something to investigate.
PDF/A is an international standard, and it's capable of holding lots of notes. Might be something to check out.
Another hint...just about every serious photo software package is based upon sqlite. Sqlite is an industrial grade database gifted to the public. Anything built on that framework may be of interest to you.
A once popular Linux product: https://wiki.gnome.org/Apps/Shotwell
Complexity and features go up from there. Windows or Linux....both based on sqlite.
You don't mention which OS. What about Photo Gallery for Windows? Lyn for Mac? Shotwell for Linux?