The project settings on the right are set and save immediately, whereas to edit things on the left, you need to press Edit Project Details first.

I clicked several times on "Default Reference Language" trying to change it before I figured out you had to click the Edit button below. Note also that button can be below the fold depending on your screen size.