What are
/r/mongodb's
favorite Products & Services?
From 3.5 billion Reddit comments
The most popular Products mentioned in /r/mongodb:
The most popular Services mentioned in /r/mongodb:
MongoDB
Studio 3T
NoSQLBooster for MongoDB
HumongouS.io
Hetzner
Robo 3T
Google Groups
freeCodeCamp
elasticsearch
Firebase
CodeSandbox
DigitalOcean
SlideShare
CouchDB
Amazon Web Services
The most popular reviews in /r/mongodb:
MongoDB is ACID-compliant and has Left Outer Joins. I'm not going to bother correcting the rest of the article.
Let me clear this up.
To use the Mongodb driver you do not have to have mongodb installed.
However you'll have to have a database running somewhere which the driver will connect to and use for storing data. You can either have a database server running on your local computer or you can run it somewhere on the cloud like https://www.mongodb.com/cloud/atlas
If you want to run the database server locally you'll have to install Mongodb.
I don't think docker has anything to do with running mongodb.
You should consider either avoiding ODM as suggested by Mongodb here: https://www.mongodb.com/developer/article/mongoose-versus-nodejs-driver/ or may like to consider a lightweight one that plays nicely with mongodb native scheme validation such as Papr https://github.com/plexinc/papr. In a performant mongo db scheme design, relation between collections should not be norm and can be handled directly without using mongoose. For relations, mongoose may generate inefficient queries (more like n+1. That being said, if you must use one then mongoose is indeed more widely used and mature.
MongoDB Enterprise Server is the distribution associated with paid subscription plans (for example, MongoDB Enterprise Advanced).
You can freely download this for evaluation and development purposes, but when doing so you also should be asked to accept the Customer Agreement.
You should read the linked agreement for full terms. The free usage clause is part of section 2 (Subscriptions):
> (b) Free Evaluation and Development. MongoDB grants you a royalty-free, nontransferable (except to your Affiliates), and nonexclusive license to use and reproduce the Software in your internal environment for evaluation and development purposes. You will not use the Software for any other purpose, including testing, quality assurance or production purposes without purchasing an Enterprise Advanced Subscription. We provide the free evaluation and development license of our Software on an “AS-IS” basis without any warranty.
There is no currently no license key or technical restrictions for the features of the Enterprise Server download, but the expectation is that you would follow the terms of the customer agreement.
I think the best way to learn Mongo is to use it - http://university.mongodb.com just last week started the M101 courses. M101J (Java) would take you through all the basics, takes only a few hours a week, and will be finished by February. Its a highly structured and absolutely excellent course to work with.
As you get comfortable with MongoDB, I'd suggest grabbing a public dataset like the ones here, importing it into MongoDB, and experimenting with queries, indexes, aggregations, geospatial queries, text search indexes, and sharding. You will learn best through experimentation and exploration of the many features.
as someone else already said, use MongoDB Atlas.
Its not expansive (there is a true free tier), very reliable (even the shared cluster), you click in your browser to manage the whole thing, and you got the support if you are in big trouble. Basically I'd recommend this solution to everyone using MongoDB, but even more for people with your tech background.
Hello. I am part of the MongoDB DevRel team. You can find some performance and feature comparison between DocumentDB and MongoDB Atlas here. I would also encourage you to sign up for Atlas and try out the free tier but keep in mind that free tier and plans under M10 are hosted in a shared environment unlike the larger plans so the performance differences will not be linear between shared and dedicated cluster plans. You can also reach out directly to the Atlas support team with any questions by clicking the chat icon on any page of the Atlas site (If you don't see it be sure to disable your ad-blocker as some of them will block required resources).
having written this I tried a few new google queries and ... duh ... found this: https://www.mongodb.com/blog/post/thinking-documents-part-1
I read the article a while back and he says:
>"The changes to MongoDB that enable multi-document transactions will not impact performance for workloads that do not require them. "
And since the dB storage engine is still wire tiger I assume no performance difference. We recently upgraded our very immature cluster to 4.0 and haven't noticed any difference in performance
What you just described is a non-trivial task for someone new to software development and setting up servers. No offense intended, but that's the impression I get of you.
People on Reddit aren't willing to hold your hand throughout doing something, especially if it's clear you need to do some learning first. You need to start by researching these concepts.
There are great tutorials online that will guide you from no knowledge at all to completing something. If you have particular problems following them, you can Google your issue (copy and paste the error message into Google, removing your own variables and stuff so that the message is generic) and you'll find blog posts and StackOverflow answers that help you solve them.
I recommend you check out the DigitalOcean tutorials. If you find them too hard to start, consider looking for an introductory course on Udemy.
You should also consider posting to the MongoDB JIRA or the google group in the community help section. Someone might be able to take a look and provide some insight into doing efficient text searching. There may be some quirks to getting max performance.
Also, what version of MongoDB are you using? I'm curious if 3.2 provides some improvements to performance if you're not using it already.
Thanks a bunch. This was exactly what I needed to know!
I found this: https://www.freecodecamp.org/news/how-to-automate-database-migrations-in-mongodb-d6b68efe084e/
And it aligns with exactly what I need to do.
For a single collection, the number of documents is not limited.
But if you are embedding fields inside a single document, there is a hard limit for it. If you are not embedding the text of the whole encyclopedia into it, it would totally be cool.
As for user related attributes, it can be totally fine to set it as sub-fields, no separate collection needed.
You can check this slide slide no.14 for schema design.
It will tell you how referencing vs embedding differ. I found it easier to just embed documents if the detail of embedded documents is not changing constantly across the entire documents, for example user details. Hope this helps!
Edit: Bible is ok to fit into the 16MB limit maybe. Don't quote me on this. Just don't get past 16MB, that's a lot
can you tell me price for a cluster with 64gb ram, 8 core cpu and 15b nvme ssd? it would be around 3500 usd in atlas.
you can setup same specs on hetzner for about 300 usd.
https://www.hetzner.com/dedicated-rootserver/ax51-nvme
and you can buy whatever support you need with rest of your money
That should definitely work; however, just know that mongodb accepts WGS84 and does not work with projected coordinated points (e.g. UTM) to my knowledge. This means that you'll have to work in Lat/Long and calculations over large areas might return weird results (because of the curvature of the earth).
You might want to also consider using leaflet to display the information. You could use it to display and interact with objects. Just don't use a background map and insert some sort of image overlay.
We've been playing around with NoSQLBooster here. It's a lot more powerful than Robo3T in my opinion, especially if you're a DBA. Robo3T still wins for quick views into the data, though. https://nosqlbooster.com/
Have you looked into Studio 3T? It's a MongoDB IDE with a SQL Import feature that supports the main SQL databases (Oracle, Microsoft SQL Server, MySQL, PostgreSQL): https://studio3t.com/knowledge-base/articles/mongodb-import-export/#import-sql-to-mongodb
I think there's two solutions to this problem and neither is a good data model. The first is just on every modification, insert a new document into the collection/table and make sure that you have a secondary index on some sort of date_modifiedfield. That way you can sort by date_modified and limit to one document for the most recent version (which you will do most of the time). This one is definitely the more simple solution.
The other is an embedded array in a document which will contain the most recent version at the end of the array. Something like this:
{ "article_name": "How to Model Your Data", "article_id": 1335, "versions" : [ {...}, {...}, {...} ] }
Also, I'm a little biased, but you should check out RethinkDB. The query language should be a bit more familiar for you coming from the RDBM world. As well, RethinkDB supports joins between collections/tables which will solve the Author <=> Article problem.
Let me know if you have any questions.
Mongo could likely do this, but this isn't its strong suit. Elasticsearch is probably a better fit here, especially if you're implementing search. There are a number of questions around using ES for books, and the official documentation even references this use case.
Hi I suggest to look there: MongoDB best practice there is article about db scheme design in mongo or clasic SQL Db it can help you learn difference.
If you are using MongoDB Atlas, you can create an Atlas Search Index that uses Lucene under the covers, which is the same search engine as elastic. there is a free tier you can experiment with.
I wrote a blog post about this a while back: https://www.mongodb.com/developer/article/mongoose-versus-nodejs-driver/
It's still pretty relevant.
At the end of the day, it really depends. I prefer the NodeJS driver personally, but using Mongoose is totally fine.
MongoDB Atlas provides single billing for your managed database clusters plus the underlying cloud provider costs. You can't apply your own AWS (or other cloud provider) credits to a MongoDB Atlas account.
If you have a generous supply of cloud provider startup credits to use, one option to consider would be managing your deployment on AWS with MongoDB Cloud Manager. Cloud Manager is a SaaS agent-based approach for monitoring, automation, and backup of a self-hosted deployment. There's a free tier for monitoring; automation & backup are paid features with a 30-day free trial.
While this approach will allow you to use your startup credits, unfortunately you'll also have to do some of the admin work to provision, secure, and scale your AWS instances.
If your application is currently able to run within the limits of the Atlas free tier (aside from data size), it also sounds like you may have modest requirements for the near term. You can find some past discussions in r/mongodb with Atlas promo codes that will help you save on costs for a paid cluster (which will go farther if you scale up conservatively). If your application needs the performance of a dedicated cluster (M10+) but doesn't need to be online 100% of the time during development, you could also consider pausing your cluster to save on costs (a paused cluster only incurs storage costs).
MongoDB supports ACID transactions when operating on either a single document or multiple documents. Anytime you're writing to one document, you can be sure of ACID guarantees. For multiple documents use the transaction syntax introduced in MongoDB 4.0.
https://www.mongodb.com/blog/post/mongodb-multi-document-acid-transactions-general-availability
This will allow you to use the multi-document approach without worrying about a scenario where one document is updated but not the other!
If you have a proper server, available on the internet (like a Web server) then you could make it available to Heroku by configuring your firewall to only accept tcp connections from Heroku IP addresses and ensuring your authentication in MongoDB is configured correctly.
But you shouldn't.
If you're looking for free database hosting for a small-ish database, then Atlas will allow you to set one up and make it available to Heroku. If you're still learning, this is what you should do.
If you don't know how to configure a server with MongoDB securely, then don't - there are many scare stories on the Internet from people who have tried that and got it wrong.
Hi there, I'm Rebecca from MongoDB. Selling these credits violates the MongoDB Startup Accelerator Terms and Conditions ("Startup package is non-transferable.") and MongoDB Cloud Terms of Service. We ask that you please immediately remove this post and any other offers to re-sell your MongoDB credits. Thank you!
I'm not sure about Mongoose specifically but in your case I think the attribute pattern would be a good place to start:
https://www.mongodb.com/blog/post/building-with-patterns-the-attribute-pattern
​
There's also the Polymorphism pattern which might help as well it's linked in the above article.
I have just done a bucket based on a unit of time like 1 hr and which sensor. Pushing each reading on to an array of values witch each having a date stamp and values. Wiretiger is copy on write anyways. Before wiretiger mongo had this blog post. https://www.mongodb.com/blog/post/schema-design-for-time-series-data-in-mongodb
Also not a bad idea to create the running stats the same time. To allow accessing stats without fetching all data points from server,
Count, sum, sum of (values)^2, min, max
> Problems with Reliability
Back in like, maybe 2.6, this was a small problem. Since 3.0 (or 2.8?) it's been a non-issue. If you're having this issue it's almost certainly because you're using Mongo incorrectly. All of that aside, it's worth noting that Mongo was not ACID compliant until recently. Mongo 4.0 introduced transactions, so this problem is now eliminated. What's more, Mongo 3.6 added a lot of improvements to how data consistency between replicas and shards works to make reading from secondaries also more reliable.
> Problems with Schema-less Design
Yes, there are issues with schema-less designs, but having a schema doesn't mean you won't have issues with it. Like all databases, you really need to plan out your structures. Mongo shifts that burden from a DB admin directly to the developers. In the case of user.email not being set, a simple null check is all you need. I can think of few modern languages that can't easily account for this case. That said, Mongo 3.6 introduced schema validation which solves this by enforcing schemas on all documents added to a collection.
What's not talked about in this article is the large number of upsides Mongo has over traditional databases, but I won't get into that too much. That is also not to say that Mongo is what everyone should pick... it's definitely not. I'd even go so far to say that unless Mongo actually fits your use case, you're probably better off with a traditional relational database.
Just got the same error while trying to install. I believe their current version is bugged or something. I just downloaded the previous stable version for a quick fix.
A couple side notes to add:
Seems potentially really expensive to have a development team primarily hit a cloud server for local development. It's worth looking into setting up a containerized approach with docker for local dev.
If the developer put 0.0.0.0 because they are having trouble finding their IP (which oddly enough, seems common), have them google "what's my ip".
If the developer has a constantly changing IP, offer to buy them a VPN service that provides a dedicated IP. My traffic is load balanced across mutliple internet providers one constantly changes my ip (starlink). To combat this, I use a dedicated ip service through NordVPN - it's about $140 USD annually.
Hah! Awesome!
You may appreciate a book my wife got for me, which should have "Don't Panic" in large friendly letters on the cover: https://www.amazon.com/Baby-Owners-Manual-Instructions-Trouble-Shooting/dp/1594745978/
I think that sums it up pretty well. The description of redis at https://redis.io/ is "Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker.". Without knowing anything about the application that you want to build, I personally wouldn't have good feeling to use Redis, that's defined as an "in-memory data structure" as the primary database for it - compared to a NoSQL database like MongoDB or a relational database like MySQL or PostgreSQL. I've been using Redis in projects as a cache (similar to memcached) besides the main database.
There's a MongoDB for Startups program (https://www.mongodb.com/startups) that might be a good fit. The program includes Atlas credits, technical advice, and some other great resources.
If your startup doesn't meet the current criteria or you'd like to discuss further, feel free to drop me a DM here (or via https://community.mongodb.com/stennie, where I check in daily) and we can discuss your situation.
https://docs.mongodb.com/manual/reference/write-concern/
> For multi-document transactions, you set the write concern at the transaction level, not at the individual operation level. Do not explicitly set the write concern for individual write operations in a transaction.
My guess is this part: .delete.one( sets the write concern on the individual operation for some reason. Is there an additional argument in the real code?
Maybe it's a problem with the Java/Scala MongoDB driver itself, that sets the write concern on the individual operations when it should not. Or maybe there is more code in the real world example. I really don't do Java.
Also, instead of the dbWithSession, dbWithTransaction methods maybe you can try the more traditional approach? (just to see if it solves the problem)
ClientSession session = client.startSession();
try {
session.startTransaction(TransactionOptions.builder().writeConcern(WriteConcern.MAJORITY).build());
const collection = db.collection[BSONCollection];
db.deleteOne(session, ...);
db.deleteOne(session, ...);
db.deleteOne(session, ...);
db.deleteOne(session, ...);
session.commitTransaction();
} catch (...)
Anyway that's really as far as I can go, for me the problem is either with the Java/Scala driver or your code.
You'll need to create an IQueryable for MongoDB and then just do a myQueryable.Count() on it. You can add Where(...) LINQ methods to it to narrow down your count query. Do not do a myCollection.ToList(), do a myCollection.AsQueryable() otherwise you'll load the entire database into memory.
https://www.mongodb.com/community/forums/t/how-to-query-collections-using-linq-methods/15946
Thanks, but I think aggregation may end up being more trouble than it's worth. I'm probably going to just redesign the schema.
For the reference of anyone else looking at this post, I posted on the official Google group for Mongo here. Apparently what I'm trying to do isn't possible (or at least without doing aggregation, as per /u/brotherwayne's suggestion), but someone from the Mongo staff had a helpful response regarding ways to redesign databases in order to address this problem.
Thanks. Following these steps I could make it work. I came up with the following bash script:
REMOTE_SSH_PORT=1234 LOCAL_PORT=2345 REMOTE_MONGO_PORT=27017 cmd="ssh -p ${REMOTE_SSH_PORT} -L ${LOCAL_PORT}:localhost:${REMOTE_MONGO_PORT} " echo "#" $cmd echo "# connect on your home machine to port ${LOCAL_PORT}" echo "# example: mongo --port ${LOCAL_PORT}" $cmd
I also wrote a blog post about it.
call the duplicated data "index" and be happy. Double linked list exist for a reason. I think realm.io and neo4j keep duplicated data under the cover.
SQL Gurus would CREATE a n:m relation table. Then people come up with hyper mathematical stuff how to store that.
https://www.mongodb.com/use-cases
From personal experience, Mongo is good for "unstructured data" applications. If a client is unable to give you a relationship structure between his models or models definition itself - MongoDB. If you have complex data with a lot of optional fields or need to save JSON objects - MongoDB.
In general, treat SQL databases as a "DB optimized for storage" (normalization, etc...) and No-SQL as "DB for CPU". Queries in No-SQL usually like a "Select X when Y" without any joins (you could use $lookup ofc) and a lot of information for these queries will be prepared in an async way, behind the curtain (by watcher or change streams)
What you have looks almost correct. The last step should be to create a new field that computes the ratio from what nb_broken and total.
db.collection.aggregate([ { $group: { _id: {'rank': '$rank', 'components': '$components'}, total: {$sum: 1}, nb_broken: { $sum: {$cond: [{$eq: ['$broken', true]}, 1, 0] } } }, }, { $project: { _id: 1, brokenRatio: { $multiply:[{ $divide: ["$nb_broken", "$total"]}, 100] } } } ])
so basically my routes?
​
https://codesandbox.io/embed/awesome-galois-9mdk2?fontsize=14&hidenavigation=1&theme=dark
It's mostly on the size of storage, but there are other information exchange costs somtimes, I think.
reach out to support I'm sure they can help you get to an estimate
also checkout https://www.mongodb.com/startups/partners?utm_campaign=startup_partner&utm_source=gan&utm_medium=referral
I just had a similar post a few days ago regarding updating a pre-existing collection.
You want to look into migration. This article I found covers how to handle doing that.
You could probably write a little script to create a collection or collections filled with don't data pretty fast, at least quicker than 15 days. Then you can see any word issues you hit with the large number of documents, check these slides out?
I feel that one big collection might be better. Maybe some embedded docs or arrays of transactions would help? Also, sharding might be easier with one big collection as well?
I'm not quite sure how this site assigns scores to different projects. For that reason I don't think this necessarily defines the best client libraries. Take http://www.findbestopensource.com/product/gett-mongojs for instance. It has a two star rating but no justification to that rating or tangible review.
If you know your index sizes you can likely determine how much memory you will need. If you are sharding this gets more complicated. Generally to determine cost with AWS you can take a look at this list and the pricing you see there you multiply by 720 to get a rough monthly cost for a single server. You likely want 3 servers minimum (1 primary, 2 secondary)
Then cost is $0.10 x GB for standard SSD storage (gp2). So for 100GB that's $10/mo per server, maybe $15 if you want some buffer space. Standard SSD scales iops based on storage size so it may be advantageous to pay for higher storage. You also can get io2 storage for higher iops, but that gets expensive fast.
You also need to keep in mind multi AZ or multi region depending on how critical your uptime is. Bandwidth between AZs / regions is not free. It's cheaper than internet bandwidth though, so depending on bandwidth costs it may make sense to host your server infrastructure in AWS as well. Data between AZs / regions is not encrypted, so you'll want to make sure your mongo servers are using ssl.
It's a little Mac OS X status bar application that wraps the MongoDB binaries. The idea is to emulate the PostgreSQL experience provided by http://postgresapp.com/ but for the MongoDB camp.
Wasn't that clear from the README? Any specific suggestions to make the description clearer? Feedback is very much welcome.
The easiest way to do this would be to install Docker for desktop, then follow the instructions from github, other than that without any PHP experience it would be a long and difficult learning curve.
Noticed you guys are using a GIF on the homepage for your animation. Have you checked out Rive (https://rive.app)? It’s a freemium tool for creating quality, performant animations which can be embedded on both the web and various mobile app platforms.
Get a decent mongo GUI tool like robo3t or studio3t. These tools have built in script editors to make it easier to write and save js that you want to run.
Also your question needs better clarification. If you just want to run a .js file, you can run mongo localhost somefile.js
My favorite is definitely humongous.io. They will certainly support v3 of mongoDB since they're web-based. But I don't really plan on upgrading my prod BD on mongoDB 3.0 anytime soon. I haven't taken a look at the release changes yet, but besides changes to the way it works internally, is there going to be any major API changes ?
Try googling your IP and whitelisting that instead of taking the MongoDB Atlas auto-suggestion, sometimes they aren't the same: https://studio3t.com/knowledge-base/articles/mongodb-atlas-login-ip-whitelisting/
We had this issue at my work and we basically applied logic about using the right tool for the job, and bribed them with this MongoDB IDE/GUI that we use called Studio3T: https://studio3t.com/
Generally the Pro Mongo devs just point out which use cases are a better fit for Mongo and we get them a license for Studio3T, which has a SQL Migration function which enables entire SQL databases or multiple SQL tables to be exported into a single MongoDB collection and vice versa. It also has a SQL query function which services as a translator between the two, so they don't actually need to be well versed on Mongo in the beginning. They haven't really complained when the tool is doing a lot of the leg work for them
Any database, relational or nonrelational, should be able to handle the data volume with adequate hardware. The trouble comes down to proper schema, proper query design, and indexes to support your queries.
MongoDB may work for you, though its text search leaves something to be desired. Elasticsearch is another document-oriented database that might be more appropriate.
https://www.mongodb.com/blog/post/building-with-patterns-the-outlier-pattern
Documents for one user can look like this ``` [ { "userId": 1, "name": "John Doe", "email": "", "username": "", "birthdate": "", "location": "", // a large number of general user profile fields "friendIds": [ // userIds of friends or something similar 2, 44, 55 ], "posts": [ { "createdOn": "", "text": "", "postId": "", } { "createdOn": "", "text": "", "postId": "", } { "createdOn": "", "text": "", "postId": "", } { "createdOn": "", "text": "", "postId": "", } // a large number of posts ], "hasExtras": true }, { "userId": 1, "posts": [ { "createdOn": "", "text": "", "postId": "", } { "createdOn": "", "text": "", "postId": "", } { "createdOn": "", "text": "", "postId": "", } { "createdOn": "", "text": "", "postId": "", } // a large number of posts ],
} ] ```
I read about a schema design pattern that might be of use to you.
The Outlier Pattern is an advanced pattern, but one that can result
in large performance improvements. It is frequently used in situations
when popularity is a factor, such as in social network relationships,
book sales, movie reviews, etc. The Internet has transformed our world
into a much smaller place and when something becomes popular, it
transforms the way we need to model the data around the item.
You can totally put everything in the same document. But keep an eye on the document sizes. As far as I remember, around 1-2mb is where you should start worrying.
For the map-reduce effort, I would say that Mongo's aggregation framework is lot more powerful than most people give it credit for.
Obviously the relational vs non-relational debate cannot be settled but there is nothing stopping you from using Mongo as the database for a social networking app.
I wouldn’t ‘blindly’ get rid of arrays and replace with collections. Check out the bucket pattern to see how you instead can split a document into multiple documents using upsert.
So when someone adds a reaction, you would have an update query saying ‘find post with us XXX which have less than 100 reactions’, and then ‘increment positive reaction-property, add current reaction to array of reactions, and set post if to XXX in the case of a upsert’
If there is no documents with less than 100 reactions, it will create a new document with the post id, add the reaction to the empty array and set the positive reactions to 1.
When fetching a post you will now need to do an aggregation to fetch all documents for that post and sum up the reactions.
It not wrong to split it out, but you shouldn’t do it just because you don’t know how to use common mongo patterns.
The bucket pattern also allow for easy pagination of reaction - in the case where you want to list all that have reacted. Just fetch one document at a time and you’ll have pages of 100.
https://www.mongodb.com/blog/post/building-with-patterns-the-bucket-pattern
If you're getting started with MongoDB, you might find this Node.js CRUD operations with MongoDB useful. Inserting the price in a MongoDB database should retain it even if you start your MongoDB instance. If you're not confident in hosting your own local MongoDB instance, their MongoDB Atlas might be up your alley and they have a free-tier you can use indefinitely.
This is a good fit for subset pattern. You can keep certain amount of recent orders in the restaurant collection so you can access them quickly. And keep the rest in the orders collection. https://www.mongodb.com/blog/post/building-with-patterns-the-subset-pattern
This is a good fit for subset pattern. You can keep certain amount of recent orders in the restaurant collection so you can access them quickly. And keep the rest in the orders collection. https://www.mongodb.com/blog/post/building-with-patterns-the-subset-pattern
Have you tried MongoDB's own tool: Compass? I don't know about its compatibility with older DBs, but you could look into it.
Also, what about connecting with JavaScript? This MDN tutorial (specifically Part 3) walks you through how to do that.
Hi there 👋 I'm the PM for the MongoDB Shell. Happy to hear you like it.
We'll share a bit more about this at MongoDB .Live (https://www.mongodb.com/live, make sure you attend!) in a couple of weeks but in short we will be GAing the new MongoDB Shell (mongosh) very soon and we are already almost completely backwards compatible with the existing mongo shell.
If you have questions or feedback on mongosh, feel free to DM me.
Atlas has encryption at rest and ssl on by default. That doesn’t prevent one of your users from downloading a backup and looking at your data though. Field level encryption is an extra step that encrypts a given field so only those with the key can read the field. Here is the blog post describing the feature that shows some of the use cases https://www.mongodb.com/blog/post/field-level-encryption-is-ga
There are several patterns to follow depending your use case. You can find more info here (start):
https://www.mongodb.com/blog/post/building-with-patterns-a-summary
Personally, in design phase those are the things that I try to keep in mind:
Embedded
- Small Subdocuments
- Data that does not change often
- Eventual consistency is acceptable
- Documents that grow slowly
- Data that will require more than one queries to fetch
- Fast reads
References
- Large Subdocuments
- Volatile data
- Immediate consistency is necessary
- Documents that grow fast
- Data often excluded from desired results
- Fast Writes
​
Other things to keep in mind:
- A nested (embedded) doc can go up to 100 levels
- One document can be up to 16 MB
Those two are a bit edgy, but they are limitations that we should consider.
>s but they arent a good option for my use case.
>
>how do I get something like above in Mon
you can have the top 10 messages & the other store in another collection, then you can use a join whi $lookup.
​
I recommend to you read this article https://www.mongodb.com/blog/post/building-with-patterns-the-subset-pattern
There is a modeling pattern called ‘bucketing’ : https://www.mongodb.com/blog/post/building-with-patterns-the-bucket-pattern
Basically you keep all words embedded in a array, but you split it over several documents. So instead of having one user document with 1000 words, you can have five documents all representing the same user and have 200 words in each.
I am aware of multi-document ACID transaction support. But Mongo themselves suggest that if you are using this frequently then perhaps MongoDB was a poor design decision: https://www.mongodb.com/blog/post/mongodb-qa-whats-the-deal-with-data-integrity-in-relational-databases-vs-mongodb
With the size of your sections, is there any chance you might get to the 16MB limit? Is there a lot of data in each section?
If courses usually stay well within the limit of the document size, but there might be an exotic case where a course gets much bigger than usual, consider the Outlier Pattern.
I'm sorry man but you're just taking the terminology that has been established over years of development and using it to imply that MongoDB doesn't have a certain feature.
Relational databases are databases that model data in tables with relations. MongoDB does not do that at all. Like it or not, MongoDB is a NoSQL database and thus a document database. MongoDB admits this themselves.
I know this subreddit probably isn't the right place for me to say any of this, I'll just get downvoted by fanboys that don't like hearing MongoDB's flaws, but it is what it is.
After posting, I found this article that quite nicely answer my question.
https://www.mongodb.com/blog/post/building-with-patterns-the-attribute-pattern
My main issue with having everything in the same table was the query performance with indexes.
I'm still not 100% sure how the BD will handle keeping these indexes updated when only a fraction of all entries contains all the fields.
MongoDB just merged with Realm, an offline first kinda database. You should take a look at that.
Now, I don’t know how this sync works, but I know the old mongodb stitch, which now is rebranded mongodb realm, allows for access rules so that signed in user only can read documents where a given property contains the user id etc.
With MongoDB, we indeed consider denormalization a normal thing to do. Especially if the information that is linked, like your customer_type, is purely static, or changes very infrequently. So you may indeed embed the customer_type completely into the customer object. Think about the actual size increase that would mean. Many times, it's actually negligible, and storage is cheap today.
Alternatively, you can keep customer_type as a separate collection, and do a $lookup when necessary. This is the equivalent to a JOIN, and you'd use MongoDB much like a relational database. It can do it, but the solution might not be very efficient, especially when you scale up to large amounts of data and high throughput. To mitigate that, you might choose to do the lookup on the client side, caching the values from the customer_type collection in, say, a has map, so you can fill them in whenever you retrieve a particular customer.
Another alternative is to use the Extended Reference pattern. Rather than embedding the entire customer_type document in the customer, you'd embed the type_id *and* the fields you'd most likely need from customer_type into customer – such as the name of the customer_type, and whatever else you probably need. This way, you'd avoid most of your lookups, but maybe not all of them, if you need a more exotic field from the customer_type.
You could say that MongoDB does not have less, but actually more ways of expressing relations between separate data items than a "relational" database.
Data in MongoDB still has relationships, it just isn't expressed in the same way as an RDBMS. For example, if I want to represent a 1:many relationship (ex: 1 person could have many credit cards), I would represent the many as an embedded array within the document of the one. So a person document would have an array of credit cards. The relationship still exists, it is just expressed differently.
This is a fantastic blog series for looking into this stuff in more detail: https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-1
>writing to one document, you can be sure of ACID guarantees. For multiple documents use the transaction syntax introduced in MongoDB 4.0.
>
>https://www.mongodb.com/blog/post/mongodb-multi-document-acid-transactions-general-availability
>
>This will allow you to use the multi-document approach without wor
Hi there! That feature is sweet and I'm in love with it. But the fact that mongoDB provides transactions, made it a good candidate for a Higly Transactional application?
What is your maximum number of connections configured to?
Maybe you'll find this article helpful: https://www.mongodb.com/blog/post/tuning-mongodb--linux-to-allow-for-tens-of-thousands-connections
I ran into some of these problems mentioned, particularly my mongoDb being configured to allow tens of thousands of connections, but my OS limiting it to a figure that was (I think) in the hundreds.
I don't really know much Spring, but your code looks straightforward enough, and although it is poorly formatted and hard to read, nothing glaring jumped out at me.
https://www.mongodb.com/blog/post/mongodb-go-driver-tutorial
Basics are all in there. But in reality, you will rarely be accessing data directly from the bson result, you should be unmarshalling to your own type/struct using the Decode functions.
not the reply you're looking for but you shouldn't be hosting your own database nowadays, you should probably use atlas
Thanks for getting back to me.
A few decades ago I built client side database drivers for ODBC (Microsoft’s universal DB access library). Looks like that turned into Linq. Cool.
Unless I’m confused it sounds like the problems isn’t in MongoDB, but in the MongoDB linq driver which creates a performance problem on ToList post query, whereas the SQL driver has a more sophisticated approach and doesn’t cause the problem. I googled for MongoDB C# api and it looks like MongoDB manages the driver, so they’re the best place to start.
You may not want to do this experiment, but I wonder if you’d see the same performance impact in node.js using mongoose or a sql driver and the same two backend databases. Point being if you didn’t, you’ll have isolated the problem to the drover.
Also, I’m wondering if a cursor might help here. I checked and mongodb c# does have a cursor feature. I’m unsure how you’re using the result, but perhaps the cursor code path avoids the performance impact in the ToList codepath. All of this is guessing on my part. Here’s a page I found that discusses cursors in MDB and C#.
Seriously, I’m still curious about all of this. So, if you do remember, please let me know how it goes. Good luck.
If you want to embed but is concerned about document size because of array of growing embedded documents, look up ‘Bucketing’- pattern.
Basically, when inserting the B document, use an upsert operator and search for a A document that has less than X embedded B documents already. If it finds it, it’ll add the new B to that document. If not, it’ll insert a new A document and add the B document to that instead. The new A document can be a shallow copy of the original A document. Just enough data do identify it as the correct A document.
Here’s a link to a ‘proper’ description: https://www.mongodb.com/blog/post/building-with-patterns-the-bucket-pattern
I don't see anything wrong with storing the labels in your documents. The idea with MongoDB is that it provides a lot of flexibility to make your life easier as a developer, so if storing those labels helps you reach your end goal in terms of application features then go for it!
Also, here's a blog post on MongoDB schema design that I personally found very helpful, it's worth a read: https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-1
Check out MongoDB's write performance operation documention.
> For each insert or delete write operation on a collection, MongoDB either inserts or removes the corresponding document keys from each index in the target collection. An update operation may result in updates to a subset of indexes on the collection, depending on the keys affected by the update.
Think of it as updating the index of a textbook. If I delete several references to a topic in the book, I also have to go into the index and delete those page numbers from that topic's entry. That has a performance overhead in a database.
Also check out this blog post on Performance Best Practices: Indexing. The ESR rule mentioned there is helpful when setting up compound indexes.
In terms of when to use single vs compound indexes, it really all depends on what your queries are. You should be using explain on your queries and figuring out which indexes you'll need based on that. In cases where you are doing a lot of look-ups by just one field, single indexes are all you'll need. With more complex queries or aggregations, compound indexes can be more efficient.
If you're trying to do a join, check out this blog by MongoDB and Rockset: https://www.mongodb.com/blog/post/enable-real-time-sql-rockset
Okay, fair point however this article shows a fairly similar pipeline in their example as your example. What's your thoughts? New Ways to Explore Data using MongoDB Charts
Also a flask developer, but recently scrapped mongoengine in my projects in favor of a more custom ORM-esque layer exclusively using the aggregation pipeline.
when you do references, if you build classes that use $lookup you can get the best of both worlds.
however, generally you want to think about your application usage patterns though. do you find your queries needing to span multiple collections? $lookups are less efficient regardless and i'd avoid them.
ultimatelly it comes down to application usage patterns and scale. do you envision the nested "items" array to exceed a reasonable number? then maybe it makes more sense to reference.
Here's a good blog post that goes through the scale question: https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-1
I apologize for the misunderstanding. I read it as you wanted to store the data on your own HDD so was thinking that you were asking about running a local server rather than using the MongoDB Cloud. MongoDB Atlas guarantees 99.995% uptime: https://www.mongodb.com/cloud/atlas/reliability
Do you have some estimates for your workload? Ex: reads / writes per second, total data size, etc.? If you have some ballpark numbers, this presentation is really helpful for sizing your MongoDB deployment. Info here will apply whether you end up using Atlas or self hosting MongoDB. https://www.mongodb.com/presentations/sizing-mongodb-clusters-2018
Hello. I am a member of the MongoDB DevRel team. You can learn more about the SSPL here.
In short, no. The SSPL section you mentioned applies only if you are offering MongoDB as a service (DBaaS) to customers, not when you are offering something built on top of MongoDB.
Hello. I am on the Developer Relations team at MongoDB. We have a FAQ page on the SSPL here that should help to clarify things.
From the section answering the question "What specifically is the difference between the GPL and the SSPL?" you'll find this:
>The only substantive modification is section 13, which makes clear the condition to offering MongoDB as a service. A company that offers a publicly available MongoDB as a service must release the software it uses to offer such service under the terms of the SSPL, including the management software, user interfaces, application program interfaces, automation software, monitoring software, backup software, storage software and hosting software, all such that a user could run an instance of the service using the source code made available.
Basically the only difference between the SSPL and GPL is related to offering MongoDB as a service. This doesn't affect SaaS built on top of MongoDB, only offering MongoDB itself as the service.
Note: I am not a lawyer either for you or for MongoDB and this should not be considered legal advice.
I am not a lawyer, but as far as I know, you would not have to open source your backend code, only if you modify MongoDB itself (for example, to make it more useful for video streaming) and then would offer that version of MongoDB to a third party, then you would have to open source the modifications.
"The copyleft condition of Section 13 of the SSPL applies only when you are offering the functionality of MongoDB, or modified versions of MongoDB, to third parties as a service. There is no copyleft condition for other SaaS applications that use MongoDB as a database."
https://www.mongodb.com/licensing/server-side-public-license/faq
Thanks for the reply. I read THIS and thought since they've implemented the aggregation pipeline into update the $divide and $add would work like that. Not quite sure if I can do add and divide in separate stages within update either(highly unlikely). My last option, as you suggested, will be to use $out then $merge but its a fuckin pain, cause at another point I'll need to use a conditional statement with $exists which aggregation pipeline doesnt support.
It appears from the mention of Row Based Pricing and HubSpot that /u/Hairy_Statistician is referring to StitchData which is not the same as MongoDB Stitch
> we needed to run the database cluster and OpsManager on our own infrastructure in AWS rather than using Mongo’s managed database offering. This was non-trivial, as Mongo didn’t provide any tooling for getting set up easily on AWS
MongoDB's Atlas offering allows you to deploy Mongo on your own AWS instance: https://www.mongodb.com/cloud/atlas/aws-mongodb
Just to add guys, you can also apply to MongoDB startup accelerator here - https://www.mongodb.com/startups
​
If you are successful then you can also apply to GAN and then this puts you on the path to 2 years worth of $5000 USD AWS credits.
That's interesting. Do you think it's cheaper to use VPC peering to mognodb atlas over documentdb/cosmosdb. Mongodb Atlas site price comparison seems to think so, but they might be bias a little lol. Reference: https://www.mongodb.com/atlas-vs-amazon-documentdb/pricing If I did what you're suggesting, then I guess I wouldn't need EC2 instances for express API. I could just use lambda
My guess is that the import has put the data under the wrong collection name. Try connecting to both databases with Compass and see if you can spot any differences.