XQueue Hacking

Owned by Edward Zarecor (Do Not Use) (Deactivated)
May 13, 2016

Verifying That Your Credentials Work

We provide a reference implementation application for listening to an XQueue and dispatching messages to graders.  The reference implementation, xqueue-watcher, is written in python and is open source.  You can clone or fork it from here https://github.com/edx/xqueue-watcher.  We use xqueue-watcher in production to run pull graders.  You don't have to, but, even if you don't, it's a useful tool for ensuring you can connect to your queue.  Here's a relatively quick way of verifying that you have the correct credentials and can connect.

This, of course, assumes that you have had a queue provisioned for you course either on edX infrastructure or your own.  The following commands will clone the xqueue-watcher repository, create a python virtualenvironment, install the xqueue-watcher and IPython.

$ git clone git@github.com:edx/xqueue-watcher.git
$ cd xqueue-watcher
$ virtualenv /tmp/xqueue-watcher
New python executable in /tmp/xqueue-watcher/bin/python2
Also creating executable in /tmp/xqueue-watcher/bin/python
Installing setuptools, pip...done.
$ . /tmp/xqueue-watcher/bin/activate
(xqueue-watcher) $ make requirements
(xqueue-watcher) $ pip install ipython

Assuming that all has gone well, start an IPython REPL

(xqueue-watcher) $ ipython

Type "copyright", "credits" or "license" for more information.

IPython 4.2.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [n]: # Setup logging to help figure out what, if anything, went wrong

In [n]: import logging
In [n]: logger = logging.getLogger()
In [n]: logger.setLevel(logging.DEBUG)

In [n]: # Note that this example is using both HTTP basic auth and XQueue authentication
In [n]: # for the sake of completeness.  The HTTP basic auth is encoded in URL between // and @
In [n]: # typically you will not need this.

In [n]: # Create a client a login to test you credentials
In [n]: from xqueue_watcher.client import XQueueClient
In [n]: c = XQueueClient('queue-name', xqueue_server='https://basic-auth-user:basic-auth-pass@xqueue.example.com', xqueue_auth=('xqueue-user','xqueue-pass'))
In [n]: c._login()
DEBUG:xqueue_watcher.client:Trying to login to https://basic-auth-user:basic-auth-pass@xqueue.example.com/xqueue/login/ with user: xqueue-user and pass xqueue-pass
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): xqueue.example.com
DEBUG:requests.packages.urllib3.connectionpool:"POST /xqueue/login/ HTTP/1.1" 200 61
DEBUG:xqueue_watcher.client:login response from 'https://basic-auth-user:basic-auth-pass@xqueue.example.com/xqueue/login/': {u'content': u'Logged in', u'return_code': 0}
Out[n]: True

In [n]: # Check the queue for work
In [n]: c.process_one()
INFO:requests.packages.urllib3.connectionpool:Resetting dropped connection: xqueue.example.com
DEBUG:requests.packages.urllib3.connectionpool:"GET /xqueue/get_submission/?queue_name=queue-name&block=true HTTP/1.1" 200 78
Out[n]: False

In [n]: # Things are working, but the queue is empty

If the call to _login() returns False, something is not working correctly and additional investigation is required.