...

• Huge undertaking due to wide API and potential for some direct calls in to internals that may need to be mocked out or changed
• Likely to grow in scope as new access patterns / edge cases are discovered
• Needs to be maintained along with other storage engines
• Less authentic tests, may need to beef up integration tests for explicit backend old / split mongo testing

K/V level mock

 The work here is to replace the CRUD components at a level above Mongo, the work is simpler and the results should hopefully be able the same as mongomock.

Work

• Mock out MongoKeyValueStore - 1d
• Mock out SplitMongoKVS - 1d
• Identify tests that should be mocked and add mocking to them - 3d (there are hundreds) 

Savings

• Estimated total savings across all paver tests ~280 secs, presuming we can get the same kinds of gains as we did with mongomock.

Advantages

• Still uses a lot of modulestore code, tests more authentic
• Probably saves about the same amount of time as mocking Mongo
• Much smaller API footprint to mock out than a full new backend
• No new dependencies, no libraries to fork / edit / maintain
• Conceptually simple
• Clear boundaries

Disadvantages

...

Mocking out the k/v store was suggested as a possible place to look at. Presuming I'm looking at the right places (InheritanceKeyValueStore / MongoKeyValueStore / SplitMongoKVS) it doesn't seem to be a place where we are spending much time. Further it seems like there are no clear optimizations to be made in here as the data seems to already be stored fundamentally in local memory as a dict. The time spent in all of the KVSs is somewhere in the ~10-15 second range for CMS tests. It's unclear how we could improve on that in a mock or what the gains might be.