acceptance.tests.lms.test_account_settings.AccountSettingsPageTest.test_preferred_language_field fails intermittently

Description

This test fails intermittently and has been removed from the codebase.

For triaging this bug: note that the test has been removed from the codebase, and thus the functionality is no longer covered by bok-choy test. Evaluate whether or not it is or can be covered by a lower level (e.g. python or JS unit test) and thus the test was unnecessary, or if there is now risk that a bug in the code could escape and thus the test should be fixed and re-enabled.

acceptance.tests.lms.test_account_settings.AccountSettingsPageTest.test_preferred_language_field failed in this build on jenkins and passed in this build on jenkins

The test was deleted in this PR.

Error Message

1 AssertionError: Unexpected differences found in structs: * event.old: [{'code': 'ps'}] != [] (expected != actual) * event.new: [] != [{u'code': u'ps'}] (expected != actual) Expected: {'event': {'new': [], 'old': [{'code': 'ps'}], 'setting': 'language_proficiencies', 'table': 'student_languageproficiency', 'truncated': [], 'user_id': 1094}, 'referer': u'http://localhost:8003/account/settings', 'username': 'test_497680'} Actual: {u'accept_language': u'en-US,en;q=0.5', u'agent': u'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0', u'context': {u'course_id': u'', u'org_id': u'', u'path': u'/api/user/v1/accounts/test_497680', u'user_id': 1094}, u'event': {u'new': [{u'code': u'ps'}], u'old': [], u'setting': u'language_proficiencies', u'table': u'student_languageproficiency', u'truncated': [], u'user_id': 1094L}, u'event_source': u'server', u'event_type': u'edx.user.settings.changed', u'host': u'ip-10-11-12-143.ec2.internal', u'ip': u'127.0.0.1', u'name': u'edx.user.settings.changed', u'page': None, u'referer': u'http://localhost:8003/account/settings', u'session': u'18e73cc55ea8c7111ab6b15b3ba09c57', u'time': datetime.datetime(2018, 8, 24, 17, 17, 58, 437000), u'username': u'test_497680'} Tolerating: set(['context_extra_fields', 'payload_extra_fields', 'root_extra_fields', 'string_payload'])

Stacktrace

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 self = <common.test.acceptance.tests.lms.test_account_settings.AccountSettingsPageTest testMethod=test_preferred_language_field> def test_preferred_language_field(self): """ Test behaviour of "Preferred Language" field. """ self._test_dropdown_field( u'language_proficiencies', u'Preferred Language', u'', [u'Pushto', u''], ) actual_events = self.wait_for_events(event_filter=self.settings_changed_event_filter, number_of_matches=2) self.assert_events_match( [ self.expected_settings_changed_event( 'language_proficiencies', [], [{'code': 'ps'}], table='student_languageproficiency'), self.expected_settings_changed_event( 'language_proficiencies', [{'code': 'ps'}], [], table='student_languageproficiency'), ], > actual_events ) common/test/acceptance/tests/lms/test_account_settings.py:479: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ common/test/acceptance/tests/helpers.py:662: in assert_events_match tolerate=EventMatchTolerates.lenient() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ expected = {'event': {'new': [], 'old': [{'code': 'ps'}], 'setting': 'language_proficiencies', 'table': 'student_languageproficiency', ...}, 'referer': 'http://localhost:8003/account/settings', 'username': 'test_497680'} actual = {'accept_language': 'en-US,en;q=0.5', 'agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:59.0) Gecko/20100101 Firefo...': [{'code': 'ps'}], 'old': [], 'setting': 'language_proficiencies', 'table': 'student_languageproficiency', ...}, ...} tolerate = set(['context_extra_fields', 'payload_extra_fields', 'root_extra_fields', 'string_payload']) def assert_event_matches(expected, actual, tolerate=None): """ Compare two event dictionaries. Fail if any discrepancies exist, and output the list of all discrepancies. The intent is to produce clearer error messages than "{ some massive dict } != { some other massive dict }", instead enumerating the keys that differ. Produces period separated "paths" to keys in the output, so "context.foo" refers to the following structure: { 'context': { 'foo': 'bar' # this key, value pair } } The other key difference between this comparison and `assertEquals` is that it supports differing levels of tolerance for discrepancies. We don't want to litter our tests full of exact match tests because then anytime we add a field to all events, we have to go update every single test that has a hardcoded complete event structure in it. Instead we support making partial assertions about structure and content of the event. So if I say my expected event looks like this: { 'event_type': 'foo.bar', 'event': { 'user_id': 10 } } This method will raise an assertion error if the actual event either does not contain the above fields in their exact locations in the hierarchy, or if it does contain them but has different values for them. Note that it will *not* necessarily raise an assertion error if the actual event contains other fields that are not listed in the expected event. For example, the following event would not raise an assertion error: { 'event_type': 'foo.bar', 'referer': 'http://example.com' 'event': { 'user_id': 10 } } Note that the extra "referer" field is not considered an error by default. The `tolerate` parameter takes a set that allows you to specify varying degrees of tolerance for some common eventing related issues. See the `EventMatchTolerates` class for more information about the various flags that are supported here. Example output if an error is found: Unexpected differences found in structs: * <path>: not found in actual * <path>: <expected_value> != <actual_value> (expected != actual) Expected: { <expected event } Actual: { <actual event> } "<path>" is a "." separated string indicating the key that differed. In the examples above "event.user_id" would refer to the value of the "user_id" field contained within the dictionary referred to by the "event" field in the root dictionary. """ differences = get_event_differences(expected, actual, tolerate=tolerate) if len(differences) > 0: debug_info = [ '', 'Expected:', block_indent(expected), 'Actual:', block_indent(actual), 'Tolerating:', block_indent(EventMatchTolerates.default_if_not_defined(tolerate)), ] differences = ['* ' + d for d in differences] message_lines = differences + debug_info > raise AssertionError('Unexpected differences found in structs:\n\n' + '\n'.join(message_lines)) E AssertionError: Unexpected differences found in structs: E E * event.old: [{'code': 'ps'}] != [] (expected != actual) E * event.new: [] != [{u'code': u'ps'}] (expected != actual) E E Expected: E {'event': {'new': [], E 'old': [{'code': 'ps'}], E 'setting': 'language_proficiencies', E 'table': 'student_languageproficiency', E 'truncated': [], E 'user_id': 1094}, E 'referer': u'http://localhost:8003/account/settings', E 'username': 'test_497680'} E Actual: E {u'accept_language': u'en-US,en;q=0.5', E u'agent': u'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0', E u'context': {u'course_id': u'', E u'org_id': u'', E u'path': u'/api/user/v1/accounts/test_497680', E u'user_id': 1094}, E u'event': {u'new': [{u'code': u'ps'}], E u'old': [], E u'setting': u'language_proficiencies', E u'table': u'student_languageproficiency', E u'truncated': [], E u'user_id': 1094L}, E u'event_source': u'server', E u'event_type': u'edx.user.settings.changed', E u'host': u'ip-10-11-12-143.ec2.internal', E u'ip': u'127.0.0.1', E u'name': u'edx.user.settings.changed', E u'page': None, E u'referer': u'http://localhost:8003/account/settings', E u'session': u'18e73cc55ea8c7111ab6b15b3ba09c57', E u'time': datetime.datetime(2018, 8, 24, 17, 17, 58, 437000), E u'username': u'test_497680'} E Tolerating: E set(['context_extra_fields', E 'payload_extra_fields', E 'root_extra_fields', E 'string_payload']) openedx/core/lib/tests/assertions/events.py:84: AssertionError

Steps to Reproduce

None

Current Behavior

None

Expected Behavior

None

Reason for Variance

None

Release Notes

None

User Impact Summary

None

Status

Assignee

Unassigned

Reporter

Mike Youngstrom

Labels

Reach

None

Impact

None

Customer

None

Partner Manager

None

URL

None

Contributor Name

None

Groups with Read-Only Access

None

Actual Points

None

Category of Work

None

Stakeholders

None

Priority

Unset
Configure