Publishing Configurations

Publishing Configurations#

This section documents various configurations required in QuantEcon repos along with solutions to issues we have come across in the past.

Redirects#

Redirects should be setup using sphinxext.rediraffe

The following configuration is required in _config.yml

sphinx:
  extra_extensions: [sphinxext.rediraffe]
  rediraffe_redirects:
    index_toc.md: intro.md

This will redirect an old index_toc page to new intro

Setup Cache for GitHub Actions#

Our current GitHub Actions setup adds a cache mechanism to speed up Pull Request builds.

  1. The cache.yml workflow will build the entire project when changes are made to the main branch. During this workflow a copy of the _build folder saved as a workflow artifact

  2. A PR and a publish tag will fetch the cache workflow asset and unpack the _build folder prior to running jupyter-book builds.

The _build folder includes all sphinx and jupyter-cache output so will enable full caching to prevent any unnecessary conversion to html in addition to execution

Setup for GitHub Action builds using GPU#

GitHub now offers GPU-enabled hosted runners, but they are expensive. For lectures that require GPU support for libraries such as jax or numpyro (e.g. lecture-jax, lecture-python-programming) we instead run the ci, publish, and cache workflows on RunsOn self-hosted runners on the QuantEcon AWS account, which provision the same hardware at a much lower cost.

RunsOn launches an EC2 instance on demand for each job. The runner label selects the instance type and machine image, for example:

runs-on: "runs-on=${{ github.run_id }}/family=g4dn.2xlarge/image=quantecon_ubuntu2404/disk=large"

where:

  • family=g4dn.2xlarge requests an EC2 instance with an NVIDIA Tesla T4 GPU, and

  • image=quantecon_ubuntu2404 selects our custom AMI (Ubuntu 24.04) preloaded with the NVIDIA drivers, CUDA, and cuDNN. See Build Images for how the AMI is maintained.

The GPU-enabled python stack (e.g. jax[cuda13]) is installed as a workflow step on top of the AMI, and the build output is deployed using the same publishing path as non-GPU repositories (currently peaceiris/actions-gh-pages for most repositories, with migration to the native artifact deployment provided by quantecon/actions in progress).