...
- Still uses a lot of modulestore code, tests more authentic
- Conceptually simple
- Mongomock handles almost everything we need for this
- Clear boundaries
Disadvantages
- Need to submit some PRs to, or fork, mongomock to make some necessary bug fixes (one example)
- New tests might require incur more mongomock support costs as it is still fairly young
- Leaves a lot of time on the table (modulestore, but not mongo, not in split, gridfs, etc)
- Potentially hard to segregate tests w and w/o mock
...
- Saves quite a bit more time than mocking at the split layer
- Still uses a lot of modulestore code, tests more authentic
- Conceptually simple
- Mongomock handles a lot of what we need for this
- Clear boundaries
...
- Potential savings of 10 min plus on a full paver test run
- Depending on how much savings we can squeeze from skipping various to/from Mongo transformations, and whether we can fit everything we need to in local memory
Advantages
- Saves the most test timeClear boundaries
Disadvantages
- Huge undertaking due to wide API and potential for some direct calls in to internals that may need to be mocked out or changed
- Likely to grow in scope as new access patterns / edge cases are discovered
- Needs to be maintained along with other storage engines
- Less authentic tests, may need to beef up integration tests for explicit backend old / split mongo testing
...