PTest

A Program to Test arbitrary other programs against a set of test data without configuration.

In essence, you provide a source file as a command line argument, and PTest tests this program against a set of test cases in a cases/ subdirectory. PTest will handle compilation and cleanup of the program (if necessary). Test case input is assumed to have a .in extension with the matching output having a .out extension (see examples/).

PTest started out as a makefile, which was messy. It then became a script, where it got its name, and sadly became limited to programming languages that can have a shebang (so no compiled languages). It then got upgraded to a "real program" with its own repository written in Common Lisp and supports single-file programs in C, C++, Common Lisp, Java, Python, Ruby.

Known Weaknesses

• Programs the output floating point values will inherently be likely to fail. I don't see a way to fix this without violating the without configuration goal of PTest.
• PTest assumes programs are self contained in a single source file. This assumption is made from the competitive programming roots of PTest.
• PTest uses cases/tmp.out as a buffer, so cases/tmp.in can never be a useful test case.
• PTest does not currently kill running programs that take too long.

Copyright/Licensing

See the LICENSE file.