Pages

Tuesday, June 7, 2011

Practical scaling and sharding for Mongo

Here are some notes I took from a great talk on "Practical scaling and sharding at MongoNYC 2011 conference" by @eliothorowitz



Scaling by optimization
- schema design
- index design
- hardware configuration

Horizontal scaling
- vertical scaling is limited
- hard to scale vertically in the cloud
- scan scale wider than higher

Replica sets
- one master at any time
- programmer determines if read hits master or a slave
- easy to setup to scale reads

Not enough
- writes don't scale
- reads are out of date on slaves
- ram/data size doesn't scale

Sharding design goals
- scale linearly
- increase capacity with no downtime (one of biggest problems with relational dbs)
- transparent to the application
- low administration to add capacity


Sharding and documents
- rich documents reduce need for joins
- no joins makes sharding solvable

Basics
- choose how you partition data (that choice is very important)
- convert from single replica set to sharding with no downtime
- full feature set
- fully consistent by default


Two ways to spread data: hash-based and range-based

Range-based.
Collection is broken into chunks by range, which default to 64 mb or 100K objects.

collection users minKey { name: 'Miller' } maxKey { name : 'Nessman' } location shard2
collection users minKey { name: 'Nessman'} maxKey { name : 'Ogden'} location shard1

Choosing a shard-key
- shard key determines how data is partitioned
- hard to change
- most important performance decision.


Use case: photos
{ photo_id : ????, data: }
What's the right key?
- auto-increment
- md5(data)
- month() + md5(data)

Initial loading
- system starts with 1 chunk
- writes will hit 1 shard then move
- presplitting for initial bulk loading can dramatically improve bulk load time

Administering a cluster:
- do not wait too long to add capacity
- need capacity for normal workload + cost of moving data
- stay < 70% operational capacity



Reactions:

3 comments:

Occasionally, you cannot change fossil fuel right into a gemstone regardless of just how much replica cartier watches as well as warmth a person utilize.
It isn't which somebody needs to be delivered to market, however when they do not have the actual natural expertise and wish to function difficult, these people don't carry out.
This particular is among the in history faves simply because chanel replica sale is simply simple to repair. Businesses which do not monitor their own possibilities on the system that's entirely possible that Product sales to understand they're not really striking their own focuses on tend to be requesting failing.
This really is absolutely no distinct from utilizing a chanel replica sale. If you do not understand in which you are, exactly where you have to proceed, the time you've arrive, as well as ways to get presently there, how could you actually achieve your own location. Product sales 's no various. Obtain correct hermes replica sale in position as well as at this time, you will find a lot of affordable choices.

Everyone like to use iPhone mobiles and other technology but their security is very high because i have experience of using their machine in my college paper writing services company. We can get information by this post about most common iPhone pass codes.

Post a Comment