diff --git a/.gitignore b/.gitignore index 2980344..59954bb 100644 --- a/.gitignore +++ b/.gitignore @@ -5,12 +5,11 @@ !*.c !*.h +!*.rst !LICENSE !HEADER -!README.md !ChangeLog -!doc/* !Makefile.am !configure.ac diff --git a/README.md b/README.md index 3d367c9..d04edfb 100644 --- a/README.md +++ b/README.md @@ -28,30 +28,19 @@ the user would have with other frameworks: reported and tested. * Progress and statistics can be followed in real time with report hooks. -## Installation +## Documentation -```bash -$ git clone https://github.com/Snaipe/Criterion.git -$ cd Criterion -$ ./autogen.sh && ./configure && make && sudo make install -``` - -## Usage - -Given a test file named test.c, compile it with `-lcriterion`: - -```bash -$ gcc -o test test.c -lcriterion -``` +An online documentation is available on [ReadTheDocs][online-docs] +([PDF][pdf-docs] | [Zip][zip-docs] | [Epub][epub-docs]) ## Samples -Sample tests can be found in the [sample directory](https://github.com/Snaipe/Criterion/tree/master/samples). +Sample tests can be found in the [sample directory][samples]. -* [A simple test](https://github.com/Snaipe/Criterion/blob/master/samples/simple.c) -* [Using multiple suites](https://github.com/Snaipe/Criterion/blob/master/samples/suites.c) -* [Tests with signals](https://github.com/Snaipe/Criterion/blob/master/samples/signal.c) -* [Using report hooks](https://github.com/Snaipe/Criterion/blob/master/samples/report.c) +* [A simple test][sample-simple] +* [Using multiple suites][sample-suites] +* [Tests with signals][sample-signal] +* [Using report hooks][sample-report] ## F.A.Q. @@ -69,6 +58,17 @@ A. Currently, on Linux 2.6.32 and Linux 3.15.7, although it should work on most \*nix systems. More tests will be added on the build matrix. **Q. Will this work under Windows/MSVC?** -A. Windows support with MinGW is coming, but MSVC is a bit of a lost cause - for the C language. The project internally uses c99 features and gnu - extensions, and MSVC is stuck at supporting c89. +A. Windows support with MinGW/MSVC is coming, but MSVC is a bit of a lost cause + to compile the library itself: the project internally uses c99 features and gnu + extensions. + +[online-docs]: http://criterion.readthedocs.org/ +[pdf-docs]: http://readthedocs.org/projects/criterion/downloads/pdf/latest/ +[zip-docs]: http://readthedocs.org/projects/criterion/downloads/htmlzip/latest/ +[epub-docs]: http://readthedocs.org/projects/criterion/downloads/epub/latest/ + +[samples]: https://github.com/Snaipe/Criterion/tree/master/samples +[sample-simple]: https://github.com/Snaipe/Criterion/blob/master/samples/simple.c +[sample-suites]: https://github.com/Snaipe/Criterion/blob/master/samples/suites.c +[sample-signal]: https://github.com/Snaipe/Criterion/blob/master/samples/signal.c +[sample-report]: https://github.com/Snaipe/Criterion/blob/master/samples/report.c diff --git a/doc/conf.py b/doc/conf.py new file mode 100644 index 0000000..5cb2107 --- /dev/null +++ b/doc/conf.py @@ -0,0 +1,292 @@ +# -*- coding: utf-8 -*- +import sys +import os + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +#sys.path.insert(0, os.path.abspath('.')) +sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "_ext"))) + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +#needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix of source filenames. +source_suffix = '.rst' + +# The encoding of source files. +#source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = u'Criterion' +copyright = u'2015, Franklin "Snaipe" Mathieu' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version = '0.1.0' +# The full version, including alpha/beta/rc tags. +release = version + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +#language = None + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +#today = '' +# Else, today_fmt is used as the format for a strftime call. +#today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = ['_build', 'html', 'doctrees'] + +# The reST default role (used for this markup: `text`) to use for all +# documents. +#default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +#add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +#add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +#show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# A list of ignored prefixes for module index sorting. +#modindex_common_prefix = [] + +# If true, keep warnings as "system message" paragraphs in the built documents. +#keep_warnings = False + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = 'default' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +#html_theme_options = {} + +# Add any paths that contain custom themes here, relative to this directory. +#html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# " v documentation". +#html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +#html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +#html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +#html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +# Add any extra paths that contain custom files (such as robots.txt or +# .htaccess) here, relative to this directory. These files are copied +# directly to the root of the documentation. +#html_extra_path = [] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +#html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +#html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +#html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +#html_additional_pages = {} + +# If false, no module index is generated. +#html_domain_indices = True + +# If false, no index is generated. +#html_use_index = True + +# If true, the index is split into individual pages for each letter. +#html_split_index = False + +# If true, links to the reST sources are added to the pages. +#html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +#html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +#html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +#html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +#html_file_suffix = None + +# Output file base name for HTML help builder. +htmlhelp_basename = 'CriterionDoc' + + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { +# The paper size ('letterpaper' or 'a4paper'). +#'papersize': 'letterpaper', + +# The font size ('10pt', '11pt' or '12pt'). +#'pointsize': '10pt', + +# Additional stuff for the LaTeX preamble. +#'preamble': '', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + ('index', 'criterion.tex', u'Criterion Documentation', + u'Franklin "Snaipe" Mathieu', 'manual'), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +#latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +#latex_use_parts = False + +# If true, show page references after internal links. +#latex_show_pagerefs = False + +# If true, show URL addresses after external links. +#latex_show_urls = False + +# Documents to append as an appendix to all manuals. +#latex_appendices = [] + +# If false, no module index is generated. +#latex_domain_indices = True + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + ('index', 'criterion', u'Criterion Documentation', + [u'Franklin "Snaipe" Mathieu'], 3) +] + +# If true, show URL addresses after external links. +#man_show_urls = False + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + ('index', 'Criterion', u'Criterion Documentation', + u'Criterion', 'Criterion', '', + ''), +] + +# Documents to append as an appendix to all manuals. +#texinfo_appendices = [] + +# If false, no module index is generated. +#texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +#texinfo_show_urls = 'footnote' + + +# -- Options for Epub output --------------------------------------------------- + +# Bibliographic Dublin Core info. +epub_title = u'Criterion' +epub_author = u'Franklin "Snaipe" Mathieu' +epub_publisher = u'Franklin "Snaipe" Mathieu' +epub_copyright = copyright + +# The language of the text. It defaults to the language option +# or en if the language is not set. +#epub_language = '' + +# The scheme of the identifier. Typical schemes are ISBN or URL. +#epub_scheme = '' + +# The unique identifier of the text. This can be a ISBN number +# or the project homepage. +#epub_identifier = '' + +# A unique identification for the text. +#epub_uid = '' + +# A tuple containing the cover image and cover page html template filenames. +#epub_cover = () + +# HTML files that should be inserted before the pages created by sphinx. +# The format is a list of tuples containing the path and title. +#epub_pre_files = [] + +# HTML files shat should be inserted after the pages created by sphinx. +# The format is a list of tuples containing the path and title. +#epub_post_files = [] + +# A list of files that should not be packed into the epub file. +#epub_exclude_files = [] + +# The depth of the table of contents in toc.ncx. +#epub_tocdepth = 3 + +# Allow duplicate toc entries. +#epub_tocdup = True + +# Highlight PHP without starting + #include + + ReportHook(Phase)() { + } + +The macro takes a Phase parameter that indicates the phase at which the function +shall be run. Valid phases are described below. + +Testing Phases +-------------- + +The flow of the test process goes as follows: + +1. ``PRE_EVERYTHING``: occurs before running the tests. +#. ``PRE_INIT``: occurs before a test is initialized. +#. ``PRE_TEST``: occurs after the test initialization, but before the test is run. +#. ``ASSERT``: occurs when an assertion is hit +#. ``TEST_CRASH``: occurs when a test crashes unexpectedly. +#. ``POST_TEST``: occurs after a test ends, but before the test finalization. +#. ``POST_FINI``: occurs after a test finalization. +#. ``POST_EVERYTHING``: occurs after all the tests are done. + +Hook Parameters +--------------- + +A report hook may take zero or one parameter. If a parameter is given, it +is undefined behaviour if it is not a pointer type and not of the proper pointed +type for that phase. + +Valid types for each phases are: + +* ``struct criterion_test *`` for ``PRE_INIT`` and ``PRE_TEST``. +* ``struct criterion_test_stats *`` for ``POST_TEST``, ``POST_FINI``, and ``TEST_CRASH``. +* ``struct criterion_assert_stats *`` for ``ASSERT``. +* ``struct criterion_global_stats *`` for ``POST_EVERYTHING``. + +``PRE_EVERYTHING`` does not take any parameter. diff --git a/doc/index.rst b/doc/index.rst new file mode 100644 index 0000000..b138762 --- /dev/null +++ b/doc/index.rst @@ -0,0 +1,11 @@ +Criterion +========= + + .. toctree:: + :maxdepth: 2 + + intro + setup + starter + hooks + env diff --git a/doc/intro.rst b/doc/intro.rst new file mode 100644 index 0000000..e26f031 --- /dev/null +++ b/doc/intro.rst @@ -0,0 +1,28 @@ +Introduction +============ + +Criterion is a dead-simple, non-intrusive testing framework for the C +programming language. + +Philosophy +---------- + +Most test frameworks for C require a lot of boilerplate code to +set up tests and test suites -- you need to create a main, +then register new test suites, then register the tests within +these suits, and finally call the right functions. + +This gives the user great control, at the unfortunate cost of simplicity. + +Criterion follows the KISS principle, while keeping the control +the user would have with other frameworks. + +Features +-------- + +* Tests are automatically registered when declared. +* A default entry point is provided, no need to declare a main + unless you want to do special handling. +* Test are isolated in their own process, crashes and signals can be + reported and tested. +* Progress and statistics can be followed in real time with report hooks. diff --git a/doc/setup.rst b/doc/setup.rst new file mode 100644 index 0000000..aa26708 --- /dev/null +++ b/doc/setup.rst @@ -0,0 +1,29 @@ +Setup +===== + +Prerequisites +------------- + +Currently, this library only works under \*nix systems. + +To compile the static library and its dependencies, GCC 4.9+ is needed. + +To use the static library, GCC or Clang are needed. + +Installation +------------ + +.. code-block:: bash + + $ git clone https://github.com/Snaipe/Criterion.git + $ cd Criterion + $ ./autogen.sh && ./configure && make && sudo make install + +Usage +----- + +Given a test file named test.c, compile it with `-lcriterion`: + +.. code-block:: bash + + $ gcc -o test test.c -lcriterion diff --git a/doc/starter.rst b/doc/starter.rst new file mode 100644 index 0000000..7b17da6 --- /dev/null +++ b/doc/starter.rst @@ -0,0 +1,106 @@ +Getting started +=============== + +Adding tests +------------ + +Adding tests is done using the ``Test`` macro: + +.. code-block:: c + + #include + + Test(suite_name, test_name) { + // test contents + } + +``suite_name`` and ``test_name`` are the identifiers of the test suite and +the test, respectively. These identifiers must follow the language +identifier format. + +Tests are automatically sorted by suite, then by name using the alphabetical +order. + +Asserting things +---------------- + +Assertions come in two kinds: + +* ``assert*`` are assertions that are fatal to the current test if failed; + in other words, if the condition evaluates to ``false``, the test is + marked as a failure and the execution of the function is aborted. +* ``expect*`` are, in the other hand, assertions that are not fatal to the + test. Execution will continue even if the condition evaluates to + ``false``, but the test will be marked as a failure. + +``assert()`` and ``expect()`` are the most simple kinds of assertions +criterion has to offer. They both take a mandatory condition as a first +parameter, and an optional failure message: + +.. code-block:: c + + #include + #include + + Test(sample, test) { + expect(strlen("Test") == 4, "Expected \"Test\" to have a length of 4."); + expect(strlen("Hello") == 4, "This will always fail, why did I add this?"); + assert(strlen("") == 0); + } + +On top of those, more assertions are available for common operations: + +* ``{assert,expect}Equal(Actual, Expected, [Message])`` +* ``{assert,expect}NotEqual(Actual, Unexpected, [Message])`` +* ``{assert,expect}StringsEqual(Actual, Expected, [Message])`` +* ``{assert,expect}StringsNotEqual(Actual, Unexpected, [Message])`` +* ``{assert,expect}ArraysEqual(Actual, Expected, Size, [Message])`` +* ``{assert,expect}ArraysNotEqual(Actual, Unexpected, Size, [Message])`` + +Initialization and finalization +------------------------------- + +Tests that need some setup and teardown can register functions that will +run before and after the test function: + +.. code-block:: c + + #include + #include + + void setup(void) { + puts("Runs before the test"); + } + + void teardown(void) { + puts("Runs after the test"); + } + + Test(suite_name, test_name, .init = setup, .fini = teardown) { + // test contents + } + +Testing signals +--------------- + +If a test receives a signal, it will by default be marked as a failure. +You can, however, expect a test to only pass if a special kind of signal +is received: + +.. code-block:: c + + #include + #include + #include + + // This test will fail + Test(sample, failing) { + int *ptr = NULL; + *ptr = 42; + } + + // This test will pass + Test(sample, passing, .signal = SIGSEGV) { + int *ptr = NULL; + *ptr = 42; + }