Python 3 Conversion and Usage Tips
Design Principles
- Convert to bytes closest to the perimeter.
- For example, pass strings around within the code as they were before, but convert them to bytes just when serializing (e.g., persisting them to a CSV or sending them over the wire). Here's an example of where I would suggest having
send_response
take a string instead of bytes: https://github.com/edx/edx-platform/blob/a9d4f3c972223b502ef1addd6621e31fa49e61e7/common/djangoapps/terrain/stubs/youtube.py#L112-L114 - From OEP-7: Bytes should only be used when you can answer the question: “Do I need this specific sequence of bytes.” The most error-resistant way to achieve this is to use what is called a “unicode sandwich.” This means that as soon as you receive data from a file or network interface, it should be converted to text. Your code should then treat it as text for as long as possible, only encoding it back to bytes when sending it to an interface that requires bytes (such as a file, a network interface, or a bytes-oriented library).
- For example, pass strings around within the code as they were before, but convert them to bytes just when serializing (e.g., persisting them to a CSV or sending them over the wire). Here's an example of where I would suggest having
- Business logic in the code should scream back at you. Have the code be readable, so it is not swimming in details related to string conversions.
- For example, use
self.assertContains
andself.assertNotContains
instead of convertingresponse.content
frombytes
. Reason:self.assertContains
andself.assertNotContains
do the appropriate translations for us and they read much better.
- For example, use
Efficiency Tricks
- Use Regex replacements whenever possible - and bulk update the entire codebase. PR 21842 is an example doing this for all
assertIn(..response.content)
cases. - Since test startup time is slow, run and fix all tests within a file in bulk - rather than one test at a time.
FAQ
Question | Answer | |
---|---|---|
1 | What version of Python 3 should our code work on? | For webservices 3.5, For libraries, 3.5 and 3.6 |
2 | How do I indicate that a repo works with Python 3? | Make sure that your openedx.yaml file has an up-to-date oeps section that indicates that your repo is OEP-7 compliant. See here for an example. |
3 | Invalid Syntax ur"regex_string" | The If the string passes in other strings via to do this before interpolating them into your unicode regex string. |
4 | Sorted with cmp seems to be broken? | The cmp argument of sorted goes away in python 3. You need to change how the sorting is done to be able to generate a sane order using the key function instead. |
5 | response.content is now a bytestring | Option A (when using Django.test): Convert any Option B (if not using Django.test): Convert string to be a bytestring. Example PR: TODO |
6 | Output of random is inconsistent across Python versions | Option A (when no compatibility issues): Use Python3's Option B (when compatibility issues exist): If there are any data compatibility issues because of legacy code that assumes randomization consistency, use the random2 library in order to stick with Python2's old implementation. A situation where this might happen is if we're only storing a seed and recompute the state of an XBlock based on that seed using random.choice/shuffle/etc. as happens with Capa problems. See this thread. |