My Dream GHC Environment: A Few Things We Do Right, and Many We Do Wrong

Curt Sampson

Starling Software, Tokyo, Japan

Presentation Roadmap

1. QAM, my devleop/build/test/deployment system
2. What a typical development session looks like
3. Key points about the environment
4. GHC Tricks
5. Notes on the installation of the Haskell Platform
6. Summary

QAM 1

• Framework for building, testing and deploying complex, multi-program applications
• Written in Ruby, for portability. Has been known to bootstrap itself.
• Not Haskell-specific; has been used for, e.g., PHP/Apache web sites

QAM 2

• Installs into its own release dirs, not OS dirs. Single build dir.
• Extensive support for unit, functional and whole-system testing, including DBMS testing
• Encourages building external dependencies in QAM.
• http://www.starling-software.com/en/qam.html

Developer's Day

1. New Checkout, Build, Test
2. Hack, Hack, Hack
3. Time to Commit
4. Release

Developer's Day: New Checkout

• Builds "extsrc" programs/libs (lighttpd, Data.Binary).
• Builds my libraries and programs.
• I find out if there's anything I need from the system that's not installed.
• Automatically finds and runs unit tests for all languages.
• Runs functional tests.
• Runs system test (e.g., load db, start web server, http requests)

Developer's Day: Hack, Hack, Hack

• Agile-style development
• Add a unit test. Load module in interpreter and fail to compile. This uses compiled versions of modules from step 1, only interpreting the module I've changed.
• Hack a bit, re-load in interpreter, repeat this step half a dozen times per minute. Speed of rebuilding (and possibly running) is important here!
• Debugging: move around in the interpreter to different modules to play with those functions and make sure they're doing the right thing.

Developer's Day: Time to Commit

• Compile, fix the warnings that the interpreter wasn't giving me.
• Run selected functional tests (not usually all of them). Fix failures.
• Commit, or do more testing, up to the full test suite if I feel nervous.
• If I'm uncertain of my build, remove part or all of the build and release dirs to get a clean build.

Developer's Day: Release

• Release is just the same, except without the "hack, hack, hack"!
• Production system runs from a checkout just like a developer uses.
• Often will do a test run in a "developer" or "staging" checkout on the production machine before touching the production checkout.

Key Points: Development

• Minimal rebuilds: changing one source file rebuilds just one object file, and only dependent executables
• Easy and reliable way to clean for a full rebuild when necessary.
• Support for partial interpretation in the development environment.
• Selective running of tests.
• Ever-expanding cycles of builds, from small to large.

Key Points: Deployment

• Projects tend to rely minimally on OS libs/programs, and bring their own along. We don't do "cabal-install."
• Checkouts are independent and can be used concurrently.
• Development and production deployment is identical.
• Lots of support for sophisticated testing.

GHC Tricks

• Keep unit tests in the same file, but don't export them. Interpreters run the tests; compiled builds exclude them.
• main function in library modules (well, not quite yet—needs separate compilation of these).

The Haskell Platform

• Good stuff, provides that base of things oft-used and cumbersome to build for every checkout.
• Need to be able to have several versions installed on development machines.
• Should install easily outside of OS packaging system.
• Ideally, I'd build it from QAM extsrc, but it's just too big.

Summary

• Configuration control is a pain: keep as much of it in the project as you can.
• Also work hard to make development fast and easy.
• Development and production environments should be the same, as far as possible.
• Developers should do deployment.
• Conclusion: a project should be your own personal "virtual LISP machine."
• http://www.starling-software.com/en/qam.html