This document details the design of Quizzera’s infrastructure as well as the components used within. The infrastructure is designed to provide high levels of abstraction for developing and maintaining new questions. In particular, there are a lot of inner workings that are hidden from developers who focus solely on these questions. This document, then, describes those inner workings.
Quizzera is written in Django and utilizes Django REST Framework heavily for its API. It runs on Python 3.5+ and requires numerous backing services.
The “core” of Quizzera is focused around loading question modules, managing the interaction of users (and backend services) with those question modules, and caching pre-generated instances. In addition, the “core” is where the standard URLs and views live. All user-related requests go through the core views.
The primary interfaces for defining question modules (which are described in more depth in the Question Modules Guide) is also located here.
4.3.1. Question Loader¶
QuestionLoader is responsible for taking a question provider (represented
as a string, such as ‘AcyclicShortestPathsExercise’) and returning the
appropriate question module, either as a loaded package or as a class within
that package. These packages are imported from a specified set of filepaths
QuestionLoader will cache loaded packages (in-memory) to avoid
repeatedly importing packages. The cache can be disabled, but should only be
done so for development; in production, the cache should remain enabled.
|note:||Even with a disabled cache, question modules will not be reloaded dynamically; this is because Python internally caches imported packages. As a consequence, the Python internal cache would also need to be cleared (for a specific package) for code changes within a package to affect a running instance of Quizzera.|
The loading functionality also supports finding all packages along the search paths. Generally, this is used for “bootstrapping” the cache so that the first searches for a question module do not incur any overhead other than simply performing an in-memory lookup.
4.3.2. Question Manager¶
QuestionManager is responsible for taking requests and performing the
necessary operations using the appropriate question module. More specifically,
request handlers will hand off tasks to the
QuestionManager to interact with
the question modules.
It handles starting new attempts on a question, submitting attempts, and viewing attempts. In addition, it also performs all validation of attempts, primarily through using the interface exposed by all question modules.
4.3.3. Cache Manager¶
CacheManager deals with supplying new instances for questions upon
request. It does so by generating instances for questions, both on-demand and
periodically on a schedule. It acts as a look-through cache, in that any time
a new question instance for a particular provider is required, it goes through
CacheManager. Internally, it will first check the database cache to see
if there are stored instances of the requested provider. If not, it will
generate a new instance on-demand.
Ideally, there are always instances stored in the cache. This greatly speeds up
obtaining a new instance (which is especially important for user-made requests)
as it requires only a database query instead of a executing complex code. Some
of the question modules can be incredibly complex or incur additional network
requests, so generating these offline is vital for minimizing request latency.
Thus, instances of every known provider are generated periodically and stored
in the cache’s database. The number of instances per provider cached is
settings.QUIZZERA['INSTANCE_CACHE_SIZE']. The cache size should
be adjusted depending on the request rate expected.
Quizzera’s API handles all requests, including those for student attempts. The
API is designed with Django REST Framework, which minimizes the boilerplate
code required. In addition, the API processes every request involved with any
data in the database. No external access to data is permitted unless it goes
through the API; this allows us to enforce a set of restrictions easily.
As a consequence, the API manages all permission handling; this is its most
vital job (in addition to, of course, serving and receiving data).
The permissions, serialization, and URL routing is all handled by Django
REST Framework. Consequently, the actual implementation of the API is generally
minimal except for custom endpoints. Custom endpoints generally include that
for performing bulk operations (such as exporting grades or enrolling users),
which are not defined by default in Django REST Framework. In addition, most of
the mechanisms for how question attempts are handled are overridden to proxy
the requests to the
QuestionManager, as needed.
The actual usage of the API is described in more detail in the REST API
documentation. The API is versioned, though currently only a single version,
v1, is available. The versioning is to allow for future modifications to the
API without immediately breaking older functionality.
The API is authenticated using WSSE; this is a simple authentication mechanism
that we have found effective for such a project. Our WSSE implementation is
located on GitHub. Example
usage is also listed there. Any user can automatically obtain a key for the API
/api/v1/keys page. This key authenticates 1-to-1 with their user,
so the key has the same permissions as the user to whom it belongs.
Authenticating to the general website is done with CAS, the Central Authentication System. This service is provided by Princeton University and allows any member of the university to use Quizzera.
Celery is used to schedule background tasks. Currently, this is only used for repopulating the question instance cache, which is done periodically.
Tests are run through the Nose test runner. In general, everything is tested to
the extent possible. Tests are nested under the respective Django app or
subdirectory. For example, the tests for the
core app are located at
Quizgen is a Java library developed within the Computer Science department at Princeton that dynamically genrates instances of various questions used in the popular Data Structures and Algorithms course, COS 226. Because a lot of the content is already systemized in this library, Quizzera simply extends its functionality and makes use of its content. In particular, all of the question generation for older questions are done using Quizgen.
Quizzera interfaces with Quizgen using an API. All of these interactions are
abstracted away with a standard implementation of a question module that
exposes a simple interface to obtain questions through Quizgen. Most of this
code is located in
core/quizgen. All of the question types within Quizgen
are represented as new types in Quizzera, which are then subclasses of a more