I found this battery so if I cut the connector off of this battery I can just solder the battery on my device
Do you think a LiPo battery would be better? Also, if I got a 2S charger, would I be able to charge a LiPo battery with it? I'm thinking of using two of these batteries https://www.amazon.com/EEMB-653042-Rechargeable-Connector-Certified/dp/B082152887/ref=sr_1_24?crid=HGI1A7PCQBHF&keywords=li-ion+3.7+cell&qid=1636370422&rnid=1248901011&s=hpc&sprefix=li-ion+3.7+cell%2Caps%2C62&sr=1-24
with one of these 2S chargers
https://www.amazon.com/Anmbest-Charger-Protection-Lithium-Battery/dp/B07KSQY67X/ref=sr_1_35?crid=14LNJNXMNKD3J&keywords=s2%2Bli-ion%2B7.4%2Bcharge%2Bcircuit&qid=1636366840&sprefix=s2%2Bli-ion%2B7.4%2Bcharge%2Bcircuit%2Caps%2C51&sr=8-35&th=1
As long as it supplies 3.7 volts, it's the same chemistry ( lithium ion in this case) and it fits, it's fine.
Heck,it doesn't even have to fit, if you don't mind chonk hanging out.
So, yeah. It'll be fine. You could probably even get away with using a newer higher capacity generic like this one if it fits within the dimensions of the old battery.