Improving sqlalchemy inserting performance for large set of data
# using add() for d in data: fingerprint0 = tbModel(id=d) session.add(fingerprint0) session.commit() # using bulk_save_objects dbs.bulk_save_objects([tbModel(id = d) for d in data]) dbs.commit() # using bulk_insert_mappings session.bulk_insert_mappings(tbModel,[{"id": d,} for d in data]) # using core def run(self): session.execute(tbModel.__table__.insert(),[{"id": d, } for d in data]) session.commit()
in these four way, the speed
core > bulk_insert_mappings > bulk_save_objects >>>>> add()
add is slower than others for almost 10 times
Just not use add when inserting large amount of data.