Code Example Automation

At Meta Stackoverflow there’s going to be a beta release of a Documentation feature. The idea is that Stackoverflow community members can contribute code examples for functions and classes and APIs that they use. It would be like an editable wiki cookbook of recipes, kinda like the Ruby Cookbook or the Perl Cookbook or the Python Cookbook. It’s a very good idea but since I like pushing for as much automation as possible, I started to dream about whether it would be possible to automate generating code examples. It would be hard to generate documentation that explains the code and I think that still requires human intervention but I think the problem of generating code examples is tractable.

If you have the type signatures of a set of functions you can generate a set of solutions that use the functions in different ways, constrained by whether their types match. If a function doesn’t exist that matches, an example of the type can be generated or left as a parameter (an exercise for the reader to fill ;-))

Here’s a sketch of how the idea would look like, it’s pseudo-code:

Having types to constrain the generated example helps and you definitely need to populate the examples with instances of each type. The obvious issue is that the examples generated can appear non-sensical because they don’t reflect what actual happens within each function, they treat the functions as a black-box. I’m not sure how much of an issue this would be because you’re still learning the patterns of using the functions and classes and types and you will likely be trying to run the examples yourself to see what you get.

Maybe the above example would be more accurate if I omitted the output of the evaluations and just left the calling code, like in this longer example which uses functions from two modules:

I’m not sure how much this helps either without more constraints and guidance. I still feel this is worthwhile to automate with a light editing/review pass. The goal would be to get used to different calling patterns for functions and to visualize most of the possibilities of using some library or class or function, more aimed at newbies and new users of that library/class/function.

Not sure if this is a viable path for larger code bases but there is a lot of work being done on annotating code with types, such as with TypeScript for JavaScript libraries. The wins of using types in JavaScript are so great that people are willingly adding type annotations for libraries that lack them like ReactJS and AngularJS. So it may be possible to generate large amounts of example code and have one or two humans review it and clean it up a little bit. That would be a neat project.