python - How to execute multiple update queries once in pymongo? -


i have more 100000 update queries need execute, db.collection_name.update(upsert=true) can execute 1 query statement,it slow if execute queries 1 one.

is there way collect multiple queries list execute once in pymongo?

i tried use bulk, , doesn't save time, not transaction operation :(

here code snippet:

bulk = self._db.initialize_unordered_bulk_op() user_id, result in results.items():     time_stamp = time.strftime('%y-%m-%d:%h:%m:%s')     history = {         'create_at': time_stamp,         'results': result     }     bulk.find({'user_id': user_id}).update(         {'$set': {'update_at': time_stamp}}     )     bulk.find({'user_id': user_id}).update(         {'$addtoset': {'history': history}}     ) bulk.execute() 

it same speed following update statement:

self._db.update(     {'user_id': user_id},     {'$set': {'update_at':time.strftime('%y-%m-%d:%h:%m:%s')}},      upsert=true ) self._db.update(     {'user_id': user_id},     {'$addtoset': {'history': history}},     upsert=true ) 

you introduce counter variable ensures updates sent in batches because write commands can accept no more 1000 operations there's need group operations have @ 1000 operations , re-intialise bulk operation when loop hits 1000 iteration. also, dry (don't repeat yourself): merge update statements $set , $addtoset 1 update document. final update script should perform better:

bulk = self._db.initialize_unordered_bulk_op() counter = 0;  user_id, result in results.items():     time_stamp = time.strftime('%y-%m-%d:%h:%m:%s')     history = {         'create_at': time_stamp,         'results': result     }     bulk.find({'user_id': user_id}).update({         '$set': { 'update_at': time_stamp },         '$addtoset': { 'history': history }     })     counter++      if (counter % 1000 == 0):         bulk.execute()         bulk = self._db.initialize_unordered_bulk_op()  if (counter % 1000 != 0):     bulk.execute() 

Comments

Popular posts from this blog

magento2 - Magento 2 admin grid add filter to collection -

Android volley - avoid multiple requests of the same kind to the server? -

Combining PHP Registration and Login into one class with multiple functions in one PHP file -