Bug clone - Large courses do not export

Description

This is affecting 7.012 (the MIT local version of 7.00x). It is not blocking course development at this time, but a few smaller elements of the course may have to be sacrificed.

Details from Julian's emails sent on Friday 8/30:

7.012 has been seeing some issues exporting their course from edge and, given that their course goes live pretty soon, the situation seems decently urgent. Exporting the course was working fine until Wednesday - since then, around 170MB of png images were uploaded, and now the course no longer exports. No tracebacks - the browser just times out.

I exported the pre-Wednesday version, imported it locally, added all the images, exported, and imported again into edge here. Exporting again errored out, but this time with a traceback (that hadn't happened before) – see below. gridfs seems to be asked for a file it doesn't have.

And that's my main concern: if that's the case, it might be that the export isn't the problem - in the course itself we have a reference to some file, but not the file itself. Importing the pre-Wednesday version and uploading all the assets again might not work. (Though the files themselves don't seem corrupted and don't have weird names.)

Reducing the image resolutions might help get the course out the door, but there might be some issues we're going to come across again now that courses are starting, and the import size limit has been raised, and we're making it easy to upload multiple files.

Chris - any clues as to what might be happening?

I don't think it's a browser timeout issue (Christina tried increasing the timeout by quite a bit) but it might be an nginx timeout issue.

Aug 30 21:17:34 prod-edge-edxapp-001 [service_variant=cms][root][envrod-edge-edxapp-001] ERROR [prod-edge-edxapp-001 8903] [import_export.py:283] - There was an error exporting course i4x://MITx/700/course/None.
Traceback (most recent call last):
File "/opt/wwc/edx-platform/cms/djangoapps/contentstore/views/import_export.py", line 248, in generate_export_course
export_to_xml(modulestore('direct'), contentstore(), loc, root_dir, name, modulestore())
File "/opt/wwc/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_exporter.py", line 59, in export_to_xml
contentstore.export_all_for_course(course_location, root_dir + '/' + course_dir + '/static/')
File "/opt/wwc/edx-platform/common/lib/xmodule/xmodule/contentstore/mongo.py", line 105, in export_all_for_course
self.export(asset_location, output_directory)
File "/opt/wwc/edx-platform/common/lib/xmodule/xmodule/contentstore/mongo.py", line 87, in export
content = self.find(location)
File "/opt/wwc/edx-platform/common/lib/xmodule/xmodule/contentstore/mongo.py", line 67, in find
raise NotFoundError()
NotFoundError

Steps to Reproduce

None

Current Behavior

None

Expected Behavior

None

Reason for Variance

None

Release Notes

None

User Impact Summary

None

Assignee

Unassigned

Reporter

ehuR

Labels

Reach

None

Impact

None

Platform Area

None

Customer

None

Partner Manager

None

URL

None

Contributor Name

None

Groups with Read-Only Access

None

Actual Points

None

Category of Work

None

Platform Map Area (Levels 1 & 2)

None

Platform Map Area (Levels 3 & 4)

None

Priority

CAT-3
Configure