These instructions are for installing the Runestone servers using Docker to automate much of the work involved in setting up the environment needed to run a server. If you would prefer to install all the components directly on a server yourself, see the Manual Installation instructions.
Using Docker, we can bring up the server without needing to install dependencies directly on the host. Instead, the software required to run the server will be installed in four separate Docker containers (the Runestone application, redis cache, postgres DB, and Jobe code runner).
To build a Docker application with the server and all its dependencies:
You will need to enter the root user password several times during the following steps. Look for the
Password: prompt, then enter your password. Note that no characters will be echoed when you type your password – this is a normal security precaution built into Unix.
1. Install OS-dependent prerequisites¶
Open a Terminal. In the Terminal:
/opt/homebrew/binas the first line in that file.
brew install python docker md5sha1sum.
Run the newly-installed Docker Desktop application.
Expect to wait a few minutes the first time you start the Docker Desktop. Don’t proceed until the Docker Desktop’s initialization is complete.
If running on Windows, either:
Install Ubuntu on WSL2. Next, install then run Docker Desktop; see the note above on the initialization process.
Install Ubuntu on VirtualBox or on some other virtualization software. You do not need to install the Docker Desktop.
To run in VirtualBox, configure the network settings: use NAT, select Advanced, Port forwarding, add:
Next, run the
Ubuntu program to open a Linux terminal (if using WSL2) or boot up Linux and open a terminal (for VirtualBox or other virtualization software).
Finally, follow the directions for Linux below.
If running on Linux, you must use Ubuntu 20.04 (although any version of Ubuntu 18.0+ should work). Installation on older versions of Ubuntu or other Linux distributions may require adjustments. I am a fan of digital ocean droplets and I would recommend starting out with a Docker droplet from their Marketplace. This will save you a few steps. You can of course use AWS or Linode or any other provider, but I know DO the best. Sizing Virtual Machines: You will want to make sure that you have at least 1 GB of RAM as parts of the install process do use quite a bit of memory. If you are going to use this for several classes I would recommend 2 GB. If you are teaching courses that will involve compiling Java, C, or C++ then I would recommend a configuration that includes 4 cpus as well as 4 GB of RAM.
curlby opening a terminal then typing:
sudo apt install -y curl
If this fails, run
sudo apt update then retry the command above.
2. Download the bootstrap script¶
On OS X, avoid placing your files in the Documents folder, since security features introduced in OS X 12.4 require you to give Docker additional permissions.
Download the bootstrap script. To do this, open a terminal in Ubuntu or OS X then type:
curl -fLO https://raw.githubusercontent.com/RunestoneInteractive/RunestoneServer/master/docker/docker_tools.py
3. Run the bootstrap script¶
On Windows using Ubuntu under WSL2: if you see the error message “Docker Desktop not detected…” when running either command below, but you are running the Docker Desktop, then click the gear (settings) icon in Docker Desktop, select Resources then WSL Integration, and make sure the switch next to Ubuntu is turned on.
The next step, which installs required dependencies for the remainder of the process, depends on the two mutually exclusive use cases below. Remember which use case you select (either production or development); many of the following steps vary based on your use case. Note if you are just kicking the tires on a Runestone Server you should use production not test or development. The test mode is for our testing of the server by running 100’s of unit tests.
Production use case¶
If your use case is running the server, execute:
python3 docker_tools.py init
Development use case¶
If, in addition to running the server, your use case is to change the way Runestone works or change/add to the way interactive exercises behave, then:
Fork the RunestoneServer, RunestoneComponents, and BookServer repositories. If you’ve already forked these repositories, fetch the latest updates from these upstream repositories.
In the terminal, run:
python3 docker_tools.py init --clone-rs <your Github userid>
This process may take a few minutes to complete. When it does:
Reboot your computer to update your group membership.
Run the Docker Desktop if using WSL2 on Windows or using OS X.
Open a terminal.
4. Build the necessary containers¶
In the terminal, type:
All future commands should be run in the
rsdocker/RunestoneServer directory unless instructions specify otherwise.
The next command depends on the use case you chose in the previous step.
For the production use case, execute:
For the development use case, execute:
docker-tools build --single-dev --clone-all <your Github userid>
The build will take a long time (10-20 minutes in many cases). In particular, the last line (
rm -rf $RUNESTONE_PATH) may seem to hang, but simply takes a long time to complete.
Most basic configuration can be done via two files you will need to create. These files are read every time the server is restarted - to see the effects of any changes you will need to stop the containers and restart them.
For the development use case, you do not need to modify any of the default environment variables.
For the production use case, you will need to modify these variables. To do so, edit the
.env file, which Docker will read automatically as it loads containers. A sample
.env file is provided as
./.env (copied from docker/.env.prototype on the first build). See comments in the file for details.
For the development use case, you do not need to modify any of the default Python settings.
For the production use case, you will need to modify these settings to obtain an HTTPS certificate, send the lost password e-mails, etc. These options will be in the file
models/1.py (which is automatically created on the first build).
You will NOT want to check either
models/1.py into source control, since these contain passwords. The
.gitignore file is set to ignore both of them.
6. Starting the containerized application¶
Once your environment is ready to go, you can use
docker compose to bring the containers up. This command will create four containers to run different parts of the application stack (the Runestone server, redis cache, postgres DB, jobe code testing environment).
For the production use case, execute:
docker compose up -d
This runs the container in the background (detached mode). Use
docker compose logs --followto view logging data as the container starts up and runs; open another terminal to execute instructions from the following steps.
For the development use case, execute:
docker compose up
This displays logging data from the container in the terminal. To stop the container, press ctrl-c. Open another terminal to execute the following commands.
The first time you run the command will take a lot longer as it downloads containers then installs software into the various containers. You may ignore the red message
jobe error that appears during this process. After it is complete, you can go to http://localhost/ to see the application (if you configured a hostname, substitute it for localhost). If everything so far is set up correctly, you should see a welcome/login page. Continue in the instructions to add book(s), course(s) and a user account.
rsmanage command will run many useful commands inside the container for you. With
rsmanage you can:
Add a course -
Add a user -
Get information about a course
Build a book -
rsmanage build bookname
Get a database shell in the current database -
…and many other things. Just type
rsmanage for a list of things it can do. For a list of options just type
rsmanage and the subcommand you want followed by
--help; for example,
rsmanage build --help.
7. Add books¶
No books are installed by default; you must add books using the following process. To add a book, you need to add its source code to the
RunestoneServer/books/ directory. Typically, that means cloning its source code. For example, to add
rsmanage build thinkcspy --clone https://github.com/RunestoneInteractive/thinkcspy.git
After cloning a book, you may need to add it to the database. Most of the standard books are already there, but you can use
rsmanage addcourse to add it if needed.
PreTeXt authors, see Publishing to Runestone Academy. After that, you can build a pretext book just like building a Runestone book
rsmanage build --ptx coursename
The following information applies only authoring books using the Runestone.
It is important that the folder name for the book matches the
project_name set in its
This is not always automatically the case. For example, the ThinkCPP
repository will normally be cloned into ThinkCPP but it has the
project_name set to
If there is a mismatch, you will want to rename the folder you cloned the code into so that it
Most Runestone books set
get_master_url() in their
pavement.py file. However, if the book
you are adding does not, it is critical that the
master_url variable in that file is set correctly.
If you are running docker and doing your development on the same machine then
http://localhost will work.
If you are running docker on a remote host then make sure to set it to the name of the remote host.
8. Add courses¶
To add a course based on a book, run the
rsmanage addcourse script. If you run it just like
that it will prompt you for all of the necessary details. Probably the most important thing
to point out is that if this is a new book the first time you add it you want to make sure that the basecourse and the course-name are the same. If you are creating your own course but want it
based on an existing book then make sure to use the correct base course name.
It will ask for:
Course Name: The short name to identify this course/section (do NOT include any spaces). e.g.
Base Course: The name of the book to use. This MUST match the
pavement.py of the book. e.g.
Your institution: The human readable name of your institution. e.g.
Some State U
Then you will be asked whether to allow users to access the course without logging in (defaults to yes) and whether to allow pair programming (default is no).
You do not have to restart the server to make use of the course.
Some of the default books already have “default” courses with the same name as the book. If you try to create
a course with a name like
thinkcspy you will be told that the course name is the same as the book.
9. Add users¶
To add an initial instructor account to the course you have created, you can either create a new user or add an existing user as an instructor to the course. You may use the web interface or the terminal – the command-line interface, or CLI – to do this.
Web interface: browse to
https://<your domain name>/runestone/designer/index (production use case) or
https://localhost/runestone/designer/index (development use case). Use this web page to create a new course.
CLI: To add a new user, use the
rsmanage adduser subcommand; it asks for what class to add the user to and whether or not
they should be made an instructor.
Or, if you already have an account that you want to add as an instructor to the new course, you can use the
rsmanage command to execute addinstructor which will prompt you for a username and course name:
Neither of these will require restarting the server.
Once you have logged in as an instructor, you can bulk add students through the web interface. After logging in to your running server as an instructor, browse to the Instructor’s Page, then click on the Manage Students tab.
The containerized application is configured to automatically start as soon as Docker / the Docker Desktop is started. Therefore, on OS X or Windows (when using WSL2): after a reboot or after manually shutting down the Docker Desktop, remember to start the Docker Desktop application.
Run the Docker Desktop if using WSL2 on Windows or using OS X.
Open a terminal.
At the terminal, execute
Other Tips & Tricks¶
To re-build an image:
# See the possibilities docker-tools build --help # Actually run the build (add options as desired) docker-tools build
To force a rebuild, make sure the containers are stopped by executing
docker compose stop, then rerun the build
command. The build process caches results from previous builds and should complete much more rapidly. However, the
cache can cause issues if you modify a file that the system is checking for changes. If you need to force a
complete rebuild, use:
docker-tools build -- --no-cache
You can shell into the container to look around, or otherwise test. When you enter,
you’ll be in the web2py folder, where
runestone/ is an application under
applications/. From the
RunestoneServer/ directory do:
Remember that the folder under
web2py/applications/runestone is bound to your host,
so do not edit files from inside the container otherwise they will have a change
in permissions on the host.
Data is stored on a Docker containerized application in two distinct places:
Volumes, such as the Runestone Server path (
$RUNESTONE_PATH), the BookServer path, and the Runestone Components path.
Layers in a docker image – which is everything not stored in the volumes listed above.
Anything written to layers after the Docker build process will be lost. For example, if you shell into the container then
apt install a package, these changes will be lost if the container is stopped, its configuration changed, etc. This is the nature of Docker. See the docs for more information.
To install a VNC client on Linux, execute
sudo apt install gvncviewer. Next, run
gvncviewer localhost:0 &. This allows you to open a terminal in the container, see Chrome as Selenium tests run, etc.
sudo apt install openssh-server to install a SSH server. This allows easy access from VSCode, as well as usual SSH access.
npm run build, then re-build the book (or page of a book) which uses the component you’re editing via a
runestone build or
pretext build. The unit tests do this automatically; for development, it’s easiest to make changes to the test then re-run the test to guarantee the correct builds are done.
If you make changes to the BookServer, you’ll need to stop then restart the BookServer. To do this, use
docker-tools start-servers /
If you make changes to the Runestone server, most changes will be immediately applied. However, changes in the
modules folder require a stop / start sequence to apply these changes.
You can run the unit tests in the container using the
docker-tools test command.
Testing the Entrypoint¶
If you want to test the script, the easiest thing
to do is add a command to the
docker compose to disable it, and then run commands
interactively by shelling into the container.
Bring up the containers and then shell inside. Once inside, you can then issue commands
to test the entry point script - since the other containers were started
docker compose everything in them is ready to go.
File permissions can seem a little strange when you start this container on Linux. Primarily because both
nginx and Gunicorn run as the
www-data user. So you will suddenly find your files under RunestoneServer
www-data . The container’s entry point script updates permissions to allow both you and the
container enough privileges to do your work.
Writing Your Own Book¶
PreTeXt authors, see Publishing to Runestone Academy. The following information applies only authoring books using the Runestone.
If you are writing your own book you will want to get that book set up properly in the Runestone system. You need to do the following:
Run the command
rsmanage addcourse. Use the project name you configured in
pavement.pyas the name of BOTH the course and the basecourse when it asks.
Now that your course is registered, rebuild it using the command
rsmanage build <book_name>command. If this is a PreTeXt book then build with the command
rsmanage build --ptx <book_name>where the
book_nameshould match the document-id specified in the docinfo section of the pretext book. Often found in
bookinfo.ptxbut sometimes as a peer of
main.ptxfile for the book.
If you modify the dependencies of a non-Poetry project (such as the Runestone Components or rsmanage), then
poetry update will not see these updates. To force an update, manually delete the
*.egg-info directory before running
poetry update. Note you must be in shelled in to the running docker container to run