Wake Developer Guide

This is a guide for those who wish to contribute to the compiler, which is mostly written in C++. If you wish to contribute by writing wake libraries, follow the tutorial, and either contribute to an existing wake repository or create your own and email it to michaelrfairhurst@gmail.com for inclusion/publication. For the compiler hackers, read on.

Getting the source

If you haven't used git before, install it and check out a tutorial such as this one.

The wake source is on github here, you can download it with git clone https://github.com/MichaelRFairhurst/wake-compiler.git.

Building the code

You will need gcc, make, flex, bison, nodejs, boost, xxd, and md5sum all installed on your system. Many of these are standard to unix. I have not built it with visual studio or clang before, and am not sure about how difficult it is to compile flex or bison with them. MinGW is probably best for windows, but you could try cygwin if you want. Remember that bison is just a tool, while flex is both a tool and a shared library.

You will need a compiled version of boost::test and boost::filesystem for your system. If you cannot compile boost::test, that's OK, you'll just miss out on unit tests and risk getting your pull requests turned down from missing tests or test failures. Nonetheless, travis will run these tests for you, so you can get by without it.

Now run make bin/wake. If you have everything installed and in your path, this should work just fine.

Running the tests

Try make bin/wake TEST=true to run the boost::test unit tests. Everything should work perfectly if you have boost::test installed.

To run the wake tests, which actually builds wake source code that tests the output of compilations, you'll need the latest wUnit artifacts. Put the executable wunit-compiler in your PATH (its a js file), and by default all of the .o and .table files should be in ../wUnit/bin/wakeobj and ../wUnit/bin/waketable respectively, but you can change that to whatever you'd like by setting WUNITLIBRARYROOT and WUNITTABLEROOT to something else inside wmake.mk. Now that all that has been done, you can now run make, and you should see a bunch of wake files build, then a unit test output folloowed by a happy printout of tests run vs tests failed.

Issues

Now that you have the code and/or tests building, you can pick a bug or feature from the Github issue tracker, and get to work. Feel free to ask questions before settling on anything.

I also want to call out a few milestones and labels. There are different milestones for high vs low priority, and a bucket for all the features we'd like to include. High priority or feature push are a great starting place. For labels, there is a bite-sized label for small bugs/features that are great for first taking on. Additionally, semantics, parsing, and codegen have all been split out for those who are interested/more comfortable in one area vs another. The exception to this is the fullstack label, which indicates all of the above will be involved.

Source Types in the Compiler

If you ls src, you'll see source types flex, bison, c, c++, and wake. The vast majority of the compiler lives in src/cpp. Why the others?

flex: These files contain rules for lexical analysis of wake files. One is for lexing wake source, and the other is for lexing linkable files. Lexing is also called tokenization. Tokenization breaks the text into meaningful chunks, but does not attempt to put any of them in context.

bison: Bison is a parser-generator. You can define rules, and nest rules within them. When a rule is matched, you get to call code. This code creates parse tree, where each matched rule creates a node in the tree. It uses $$ to indicate the value returned by a rule match, and it uses $1, $2, ... to indicate the value returned by the first, second, etc matched rules. Once again, we have two files, one for wake source code, and one for linkable files.

c: Bison generates c parsers, and flex generates c lexers, though there are forks that generate C++ ones. For this reason, many things related flex and bison are written in c, such as error printing, and node management.

cpp: The C++ class Parser is a wrapper for the generater bison parser. This connects the c parser to the c++ typechecker and object generator. This holds the vast majority of the code.

wake: This has subdirectories test, which contains wake test programs that assert their own valid execution to test the code generator, and std/table, and std/myobj. The tables are there for convenience. We can compile them to get their .table files, which are binary files describing their interfaces. Automatic imports like primitives and Num are serialized and compiled into the compiler source code. Others are there to expose the interface of handwritten object files that call native javacript libraries, which exist in std/myobj. These files are a temporary means of doing js interop, with a simple script to transform them into a linkable .o file.

Node and Type trees

in progress

Standard type checking

in progress

Testing

in progress