I think you should read more about how exactly docker works.
Docker containers run twice as fast as a VM, generally. You can also be running 100s of containers on the same host, and even though each container takes about 1gb of storage sapce, the layered filesystem of the docker engine means that you're probably still using just about 1gb of space. https://docs.docker.com/engine/userguide/storagedriver/imagesandcontainers/
Docker networking is even simpler, you simply expose ports on the container, and communicate to the host machine what those ports should be mapped to. Networking is literally a breeze.
As for speed, due to a shared kernel using runC containers, docker is nearly identical to native speeds.
There's very little overhead to docker memory usage, your containers will not consume any more memory than your app would running natively.
https://docs.docker.com/swarm/
As for managing that, many people, myself included, simply use swarm. Which allows you to deploy multiple containers across multiple hosts. You can very easily spin up new hosts, migrate containers between hosts, and manage resources, with very little negative impact.
I'm not talking about the images, I'm talking about the swarm discovery service.
The swarm mode token based discovery is actually using dockerhub as a key/value store. The official docs do not recommend to use it in production either, see https://docs.docker.com/swarm/discovery/#/docker-hub-as-a-hosted-discovery-service
> I can't say to Swarm on which nodes exactly I need to deploy those applications Check Swarm Filters
But it's not super clear what you want to do. Can you explain in a more specific way what you want/need? Maybe you don't even need Swarm in the end.
Thats probably a bit beyond the scope of a Reddit comment reply :D
https://docs.docker.com/swarm/
If you have used Docker, Swarm is one means of orchestrating container deployment across a cluster of Docker instances (there are other technologies that can do this too - Swarm isn't always the best option depending on the problem you are trying to solve).
You package all dependencies for that app that you're running on the container. Say you want to write to InfluxDB and then get pretty graphs in Grafana, then you would have a compose file with those being two different containers. For example: https://github.com/intelsdi-x/snap/blob/master/examples/influxdb-grafana/docker-compose.yml
Here's an example from the Docker birthday of using compose: https://github.com/docker/docker-birthday-3/tree/master/example-voting-app
For swarm, it's when you have multiple hosts. https://docs.docker.com/swarm/overview/
Same here, I've looked around for a while about scaling with databases in Docker and haven't come up with much. A lot of people seem to just not even run the database in a Docker container outside of development and use something like Ansible to deploy it instead. I'd imagine you could probably use something like Ambassadors or Zookeeper to get it all figured out but I haven't looked into it too much yet. Unfortunately doesn't seem to be a lot of info about database clusters with Docker. So far the only thing I've really come across is this which keeps the database containers on the same machine as well.
I'm not sure but I think I may be approaching the problem in a way that's contrary to how docker runs, and maybe I should be scaling hardware up instead of separating the database container/server from the other machines. If anyone has experience with this please chime in!
EDIT: So after a trip to IRC I learned that you can set labels within a swarm.
With this you can set up any number of constraints to define what servers will be for your database(s). Definitely check out the page, you can restrict deployments down to ensuring it has no existing database containers running on it already, applying groups (i.e. database_provider), etc. From that point when you have the containers on their own hardware, it essentially comes down to clustering them like you normally would.
Then, within your compose file you would just set the options as environment flags such as:
environment: # Ensure no existing database containers are running on this host - "affinity:container!=~my_db*" # Only deploy this container when the node_type (set this label when you create the host) # matches "database" - "constraint:node_type==database"