When you have a hardware wallet, you use that device to access your funds on a day-to-day basis. 99% of the time you will never lose this device, have the device stolen, drop it in a toilet, burn it in a fire, or anything else.
but....
If any of the above happen, you want to be able to access your funds without your hardware wallet. For that reason, you need to have a backup. The backup of your hardware wallet is those 24 words. If you have those words, you can access your funds at anytime. Your backup is only worthwhile if it doesn't also get destroyed with your hardware wallet.
For example, if you tape your 24 words to your hardware wallet, you drop your hardware wallet off the side of a cliff, both the device AND the words will be lost. Your backup is worthless and you have no access to your funds.
In order to prevent theft, it's probably not a good idea to keep it on you on a day-to-day basis
In order to prevent damage of an area (fire, flood, tornado), it's best to not keep the Ledger and words together, say in your desk drawer.
In order to reduce the likelihood of being stolen, it shouldn't ever be put on an electronic device (computer, phone—including taking photos of it) nor should it be kept in an area where it's at an increased risk of being stolen.
In order to prevent loss, it should be somewhere safe, where it won't accidentally be thrown away or lost.
I used to keep my backup paper wallets / keys at my parents house with other important documents (like birth certs, old tax filings, etc.). I knew it wouldn't be accidentally thrown away, those things live in a fire-proof cabinet and since they haven't been lost or destroyed in 30 years, it was a safe bet. (We moved them to a safety deposit box when we moved because I'm a grown ass independent woman (lol) who can keep track of her own shit & it was physically closer to our new place.)
Where you keep your backup is up to you. Trusted family members (preferably those who dont know what crypto is), work, storage unit, safety deposit box, buried in your friends backyard, it's all going to depend on your situation. If leaving with someone else, ensure that they understand that it's very valuable to you but only to you. If you uncle or friend decides to grow up to be a crackhead, you don't want them knowing they have access to all your money. You can say its backups of your password that house your online diary, misc. documents for work that you would be killed for if you lost, old memories that you don't want your new girlfriend finding, whatever. I originally told my parents they were old records for clients that I didn't trust myself not to lose while moving.
While you think about this, it's also not a bad time to think about other things that should probably be backed up externally. What happens if you were to get hit by a truck tomorrow? What happens if your house burns down randomly? Are there accounts, records, passwords, keys, information, documentation, etc. that you and only you have? Maybe we're just weird but I know I'm not the only one in this space that has a "if I die...." document. Because of how complicated crypto is, I have contact info for a few very trusted friends to call if I die who I trust will help my family sort the crypto side of things out. It also includes client documents and contact information because, reality is, client's don't save their shit (even though the zip is called "Dont_Lose_This-Important_Docs_for_2017_Website.zip").
You can also get something like this if you don't have any external place to keep it, or don't feel like you are that high-risk at this time (e.g. you don't have millions of dollars worth of crypto, don't have tweakers for neighbors, don't live in a place where there are random hurricanes or fires, etc.). The upside to these over traditional fireproof/waterproof safes is that they are cheaper and easier to conceal someplace random, which reduces the risk of robbery. Safes are known to house valuables. A friend recently had her house broken into and they took the entire safe because she never bolted it down. The only thing in there was some old photos of her great-grandparents and her birth certificate. That sucks.
Analyze your personal risk and your situation. Would your life be over if the funds on that device were to be lost? How upset would you be? How much money do you have stored on it? The loss created by losing access to that device (including emotional damage / stress). This is the amount of money you should spend securing it. We were recently talking and arrived at the conclusion that 100% of people who have over 1-weeks pay worth of crypto should have a hardware wallet. Considering a fire-proof bag is < $20 (aka <0.05 ETH and <0.002 BTC), it's pretty much a no brainer if you are in this space.
Hope this helps!