Inventory Management
发布于 2015-09-14 14:45:58 | 127 次阅读 | 评论: 0 | 来源: 网络整理
Overview¶
This case study provides an overview of practices and patterns for designing and developing the inventory management portions of an E-commerce application.
也可以参考
Problem¶
Customers in e-commerce stores regularly add and remove items from their “shopping cart,” change quantities multiple times, abandon the cart at any point, and sometimes have problems during and after checkout that require a hold or canceled order. These activities make it difficult to maintain inventory systems and counts and ensure that customers cannot “buy” items that are unavailable while they shop in your store.
Solution¶
This solution keeps the traditional metaphor of the shopping cart, but the shopping cart will age. After a shopping cart has been inactive for a certain period of time, all items in the cart re-enter the available inventory and the cart is empty. The state transition diagram for a shopping cart is below:
Schema¶
Inventory collections must maintain counts of the current available inventory of each stock-keeping unit (SKU; or item) as well as a list of items in carts that may return to the available inventory if they are in a shopping cart that times out. In the following example, the _id field stores the SKU:
{
_id: '00e8da9b',
qty: 16,
carted: [
{ qty: 1, cart_id: 42,
timestamp: ISODate("2012-03-09T20:55:36Z"), },
{ qty: 2, cart_id: 43,
timestamp: ISODate("2012-03-09T21:55:36Z"), },
]
}
注解
These examples use a simplified schema. In a production implementation, you may choose to merge this schema with the product catalog schema described in the “Product Catalog” document.
The SKU above has 16 items in stock, 1 item a cart, and 2 items in a second cart. This leaves a total of 19 unsold items of merchandise.
To model the shopping cart objects, you need to maintain sku, quantity, fields embedded in a shopping cart document:
{
_id: 42,
last_modified: ISODate("2012-03-09T20:55:36Z"),
status: 'active',
items: [
{ sku: '00e8da9b', qty: 1, item_details: {...} },
{ sku: '0ab42f88', qty: 4, item_details: {...} }
]
}
注解
The item_details field in each line item allows your application to display the cart contents to the user without requiring a second query to fetch details from the catalog collection.
Operations¶
This section introduces operations that you may use to support an e-commerce site. All examples in this document use the Python programming language and the PyMongo driver for MongoDB, but you can implement this system using any language you choose.
Add an Item to a Shopping Cart¶
Moving an item from the available inventory to a cart is a fundamental requirement for a shopping cart system. The most important requirement is to ensure that your application will never move an unavailable item from the inventory to the cart.
Ensure that inventory is only updated if there is sufficient inventory to satisfy the request with the following add_item_to_cart function operation.
def add_item_to_cart(cart_id, sku, qty, details):
now = datetime.utcnow()
# Make sure the cart is still active and add the line item
result = db.cart.update(
{'_id': cart_id, 'status': 'active' },
{ '$set': { 'last_modified': now },
'$push': {
'items': {'sku': sku, 'qty':qty, 'details': details } } },
w=1)
if not result['updatedExisting']:
raise CartInactive()
# Update the inventory
result = db.inventory.update(
{'_id':sku, 'qty': {'$gte': qty}},
{'$inc': {'qty': -qty},
'$push': {
'carted': { 'qty': qty, 'cart_id':cart_id,
'timestamp': now } } },
w=1)
if not result['updatedExisting']:
# Roll back our cart update
db.cart.update(
{'_id': cart_id },
{ '$pull': { 'items': {'sku': sku } } })
raise InadequateInventory()
The system does not trust that the available inventory can satisfy a request
First this operation checks to make sure that the cart is “active” before adding a item. Then, it verifies that the available inventory to satisfy the request before decrementing inventory.
If there is not adequate inventory, the system removes the cart update: by specifying w=1 and checking the result allows the application to report an error if the cart is inactive or available quantity is insufficient to satisfy the request.
注解
This operation requires no indexes beyond the default index on the _id field.
Modifying the Quantity in the Cart¶
The following process underlies adjusting the quantity of items in a users cart. The application must ensure that when a user increases the quantity of an item, in addition to updating the carted entry for the user’s cart, that the inventory exists to cover the modification.
def update_quantity(cart_id, sku, old_qty, new_qty):
now = datetime.utcnow()
delta_qty = new_qty - old_qty
# Make sure the cart is still active and add the line item
result = db.cart.update(
{'_id': cart_id, 'status': 'active', 'items.sku': sku },
{'$set': {
'last_modified': now,
'items.$.qty': new_qty },
},
w=1)
if not result['updatedExisting']:
raise CartInactive()
# Update the inventory
result = db.inventory.update(
{'_id':sku,
'carted.cart_id': cart_id,
'qty': {'$gte': delta_qty} },
{'$inc': {'qty': -delta_qty },
'$set': { 'carted.$.qty': new_qty, 'timestamp': now } },
w=1)
if not result['updatedExisting']:
# Roll back our cart update
db.cart.update(
{'_id': cart_id, 'items.sku': sku },
{'$set': { 'items.$.qty': old_qty } })
raise InadequateInventory()
注解
That the positional operator $ updates the particular carted entry and item that matched the query.
This allows the application to update the inventory and keep track of the data needed to “rollback” the cart in a single atomic operation. The code also ensures that the cart is active.
注解
This operation requires no indexes beyond the default index on the _id field.
Checking Out¶
The checkout operation must: validate the method of payment and remove the carted items after the transaction succeeds. Consider the following procedure:
def checkout(cart_id):
now = datetime.utcnow()
# Make sure the cart is still active and set to 'pending'. Also
# fetch the cart details so we can calculate the checkout price
cart = db.cart.find_and_modify(
{'_id': cart_id, 'status': 'active' },
update={'$set': { 'status': 'pending','last_modified': now } } )
if cart is None:
raise CartInactive()
# Validate payment details; collect payment
try:
collect_payment(cart)
db.cart.update(
{'_id': cart_id },
{'$set': { 'status': 'complete' } } )
db.inventory.update(
{'carted.cart_id': cart_id},
{'$pull': {'cart_id': cart_id} },
multi=True)
except:
db.cart.update(
{'_id': cart_id },
{'$set': { 'status': 'active' } } )
raise
Begin by “locking” the cart by setting its status to “pending” Then the system will verify that the cart is still active and collect payment data. Then, the findAndModify command makes it possible to update the cart atomically and return its details to capture payment information. Then:
- If the payment is successful, then the application will remove the carted items from the inventory documents and set the cart to complete.
- If payment is unsuccessful, the application will unlock the cart by setting its status to active and report a payment error.
注解
This operation requires no indexes beyond the default index on the _id field.
Returning Inventory from Timed-Out Carts¶
Process¶
Periodically, your application must “expire” inactive carts and return their items to available inventory. In the example that follows the variable timeout controls the length of time before a cart expires:
def expire_carts(timeout):
now = datetime.utcnow()
threshold = now - timedelta(seconds=timeout)
# Lock and find all the expiring carts
db.cart.update(
{'status': 'active', 'last_modified': { '$lt': threshold } },
{'$set': { 'status': 'expiring' } },
multi=True )
# Actually expire each cart
for cart in db.cart.find({'status': 'expiring'}):
# Return all line items to inventory
for item in cart['items']:
db.inventory.update(
{ '_id': item['sku'],
'carted.cart_id': cart['id'],
'carted.qty': item['qty']
},
{'$inc': { 'qty': item['qty'] },
'$pull': { 'carted': { 'cart_id': cart['id'] } } })
db.cart.update(
{'_id': cart['id'] },
{'$set': { status': 'expired' })
This procedure:
- finds all carts that are older than the threshold and are due for expiration.
- for each “expiring” cart, return all items to the available inventory.
- once the items return to the available inventory, set the status field to expired.
Indexing¶
To support returning inventory from timed-out cart, create an index to support queries on their status and last_modified fields. Use the following operations in the Python/PyMongo shell:
db.cart.ensure_index([('status', 1), ('last_modified', 1)])
Error Handling¶
The above operations do not account for one possible failure situation: if an exception occurs after updating the shopping cart but before updating the inventory collection. This would result in a shopping cart that may be absent or expired but items have not returned to available inventory.
To account for this case, your application will need a periodic cleanup operation that finds inventory items that have carted items and check that to ensure that they exist in a user’s cart, and return them to available inventory if they do not.
def cleanup_inventory(timeout):
now = datetime.utcnow()
threshold = now - timedelta(seconds=timeout)
# Find all the expiring carted items
for item in db.inventory.find(
{'carted.timestamp': {'$lt': threshold }}):
# Find all the carted items that matched
carted = dict(
(carted_item['cart_id'], carted_item)
for carted_item in item['carted']
if carted_item['timestamp'] < threshold)
# First Pass: Find any carts that are active and refresh the carted items
for cart in db.cart.find(
{ '_id': {'$in': carted.keys() },
'status':'active'}):
cart = carted[cart['_id']]
db.inventory.update(
{ '_id': item['_id'],
'carted.cart_id': cart['_id'] },
{ '$set': {'carted.$.timestamp': now } })
del carted[cart['_id']]
# Second Pass: All the carted items left in the dict need to now be
# returned to inventory
for cart_id, carted_item in carted.items():
db.inventory.update(
{ '_id': item['_id'],
'carted.cart_id': cart_id,
'carted.qty': carted_item['qty'] },
{ '$inc': { 'qty': carted_item['qty'] },
'$pull': { 'carted': { 'cart_id': cart_id } } })
To summarize: This operation finds all “carted” items that have time stamps older than the threshold. Then, the process makes two passes over these items:
- Of the items with time stamps older than the threshold, if the cart is still active, it resets the time stamp to maintain the carts.
- Of the stale items that remain in inactive carts, the operation returns these items to the inventory.
注解
The function above is safe for use because it checks to ensure that the cart has expired before returning items from the cart to inventory. However, it could be long-running and slow other updates and queries.
Use judiciously.
Sharding¶
If you need to shard the data for this system, the _id field is an ideal shard key for both carts and products because most update operations use the _id field. This allows mongos to route all updates that select on _id to a single mongod process.
There are two drawbacks for using _id as a shard key:
If the cart collection’s _id is an incrementing value, all new carts end up on a single shard.
You can mitigate this effect by choosing a random value upon the creation of a cart, such as a hash (i.e. MD5 or SHA-1) of an ObjectID, as the _id. The process for this operation would resemble the following:
import hashlib import bson cart_id = bson.ObjectId() cart_id_hash = hashlib.md5(str(cart_id)).hexdigest() cart = { "_id": cart_id, "cart_hash": cart_id_hash } db.cart.insert(cart)
Cart expiration and inventory adjustment requires update operations and queries to broadcast to all shards when using _id as a shard key.
This may be less relevant as the expiration functions run relatively infrequently and you can queue them or artificially slow them down (as with judicious use of sleep()) to minimize server load.
Use the following commands in the Python/PyMongo console to shard the cart and inventory collections:
>>> db.command('shardCollection', 'inventory'
... 'key': { '_id': 1 } )
{ "collectionsharded" : "inventory", "ok" : 1 }
>>> db.command('shardCollection', 'cart')
... 'key': { '_id': 1 } )
{ "collectionsharded" : "cart", "ok" : 1 }