This is really quite the question. You'll undoubtedly get some really insightful response from guys way smarter than me!
So let me contribute my widow's mite to the conversation. Grab a couple of books and read them slowly and thoughtfully. I think this deliberate lingering look at the subject that your post shows you've obviously given much thought to, will serve you better than a paragraph or two here in reddit - be they ever so clear!
If I may recommend two for you?
You've got to read the seminal work by O. Palmer Robertson, <em>The Christ of the Covenants</em>
Then follow that up with Michael Horton's book <em>Introducing Covenant Theology</em> for a more modern look at the subject.
Both of these you can buy used at Amazon for right around $5. I think they will be great tools for you as you work through the thoughts you outline above.
Forgive the lack of 'crunchy' in this post by skirting an answer with book recommendations!
The Christ of the Covenants
is the most recommended and it’s helpful. I also really enjoyed Covenant Theology: A Reformed Baptist Primer. I thought it did a better job of explaining the different covenants even if I disagreed with its landing on baptism.
Covenant theology is extremely helpful in that it helps connecting various things in the old testament to Christ and the new testament. Also various aspects of CT are found in early Christianity, albeit not fully developed.
Classical covenant theology (Presbyterian) has been written about extensively, but by far the best introductory book would be Christ of the Covenants by Palmer Robertson and this free online course by Ligon Duncan.
> How is the New Covenant a better Covenant (according to Hebrews 8) if it is just the same covenant as the Old but administered in a different way?
Again, the question assumes "old" means "Abrahamic", but Hebrews 8 refers to Moses, not Abraham. The old covenant in this context is referring to the Law of Moses, not the covenant God makes with Abraham. I don't know of anyone who believes the Law of Moses is the covenant of Grace. The Law was "added because of transgressions" (Gal 3:19). Check out Galatians 3 for a full explanation.
> Why are the Abrahamic and Davidic Covenants often considered unconditional when...?
Regarding Abraham, it seems to me that Paul tells us this is how we should understand the covenant. Romans 4 talks a lot about this, here's an excerpt:
> 13 For the promise to Abraham and his offspring that he would be heir of the world did not come through the law but through the righteousness of faith. 14 For if it is the adherents of the law who are to be the heirs, faith is null and the promise is void. 15 For the law brings wrath, but where there is no law there is no transgression. 16 That is why it depends on faith, in order that the promise may rest on grace and be guaranteed to all his offspring—not only to the adherent of the law but also to the one who shares the faith of Abraham, who is the father of us all, 17 as it is written, “I have made you the father of many nations”—in the presence of the God in whom he believed, who gives life to the dead and calls into existence the things that do not exist. Romans 4:13-17
Regarding David, keep in mind that 2 Sam 7 is not purely national in the immediate sense, and not purely Messianic. There are elements of both! So when God warns Solomon, this is in keeping with the covenant with David: "I will be to him a father, and he shall be to me a son. When he commits iniquity, I will discipline him with the rod of men, with the stripes of the sons of men." But the Messianic aspect is not conditioned on Solomon's obedience, or anyone else's. Note that Jeremiah explains it while telling all Judah about their impending captivity as punishment for disobedience. Here's a partial reiteration (see Jer 33 for details):
> For thus says the Lord: David shall never lack a man to sit on the throne of the house of Israel Jeremiah 33:17.
You also asked
> Why does the presence of promises of Christ and God’s Kingdom in the future necessitate the previous covenants being administrations of the Covenant of Grace?
Because of the language God uses along the way when making these covenants. God makes "an everlasting covenant" with Abraham (Gen 17:7), that Peter refers to in Acts 2:39, and Paul describes extensively in Galatians and Romans.
Also notice that each successive covenant in is always an extension of the one before it, which stresses the significance of the continuity. God tells Moses that he's bringing Israel out of Egypt as he promised, when he established the covenant with Abraham (Ex 6). David didn't just decide to build the temple, this was set up in advance by God when the Law of Moses was being delivered (Deut 12:5-18). Jesus was specifically killed at Passover and instituted the Lord's Supper at that time (Luke 22).
I highly recommend watching this video by Ligon Duncan and reading O. Palmer Robertson's excellent book <em>The Christ of the Covenants</em>. The video is about infant baptism, but he gets you there by way of his explanation of covenant theology.
I hope that's helpful!