From 7244cbfcca310a320b1f64966959b41d0dc6eb19 Mon Sep 17 00:00:00 2001 From: Snaipe Date: Mon, 9 Feb 2015 19:35:50 +0100 Subject: [PATCH 01/15] Converted README to ReStructuredText --- README.md | 74 ------------------------------------------------ README.rst | 83 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+), 74 deletions(-) delete mode 100644 README.md create mode 100644 README.rst diff --git a/README.md b/README.md deleted file mode 100644 index 3d367c9..0000000 --- a/README.md +++ /dev/null @@ -1,74 +0,0 @@ - -Criterion -========= - -[![Build Status](https://travis-ci.org/Snaipe/Criterion.svg?branch=master)](https://travis-ci.org/Snaipe/Criterion) -[![Coverage Status](https://coveralls.io/repos/Snaipe/Criterion/badge.svg?branch=master)](https://coveralls.io/r/Snaipe/Criterion?branch=master) -[![License](https://img.shields.io/badge/license-MIT-blue.svg?style=flat)](https://github.com/Snaipe/Criterion/blob/master/LICENSE) -[![Version](https://img.shields.io/github/tag/Snaipe/Criterion.svg?label=version&style=flat)](https://github.com/Snaipe/Criterion/releases) - -A dead-simple, yet extensible, C test framework. - -## Philosophy - -Most test frameworks for C require a lot of boilerplate code to -set up tests and test suites -- you need to create a main, -then register new test suites, then register the tests within -these suits, and finally call the right functions. - -This gives the user great control, at the unfortunate cost of simplicity. - -Criterion follows the KISS principle, while keeping the control -the user would have with other frameworks: - -* Tests are automatically registered when declared. -* A default entry point is provided, no need to declare a main - unless you want to do special handling. -* Test are isolated in their own process, crashes and signals can be - reported and tested. -* Progress and statistics can be followed in real time with report hooks. - -## Installation - -```bash -$ git clone https://github.com/Snaipe/Criterion.git -$ cd Criterion -$ ./autogen.sh && ./configure && make && sudo make install -``` - -## Usage - -Given a test file named test.c, compile it with `-lcriterion`: - -```bash -$ gcc -o test test.c -lcriterion -``` - -## Samples - -Sample tests can be found in the [sample directory](https://github.com/Snaipe/Criterion/tree/master/samples). - -* [A simple test](https://github.com/Snaipe/Criterion/blob/master/samples/simple.c) -* [Using multiple suites](https://github.com/Snaipe/Criterion/blob/master/samples/suites.c) -* [Tests with signals](https://github.com/Snaipe/Criterion/blob/master/samples/signal.c) -* [Using report hooks](https://github.com/Snaipe/Criterion/blob/master/samples/report.c) - -## F.A.Q. - -**Q. What's wrong with other test frameworks?** -A. I worked with CUnit and Check, and I must say that they do their job - very well -- the only thing that bugs me is that setting up a test - suite from scratch is a pain, it should really be simpler. Most - (if not all) high-level languages have test frameworks with automatic - test registration, but all the ones for C require you to set up a - main, manually register suites, then tests. Criterion tries to - fix these shortcomings. - -**Q. Where has this been tested?** -A. Currently, on Linux 2.6.32 and Linux 3.15.7, although it should work on - most \*nix systems. More tests will be added on the build matrix. - -**Q. Will this work under Windows/MSVC?** -A. Windows support with MinGW is coming, but MSVC is a bit of a lost cause - for the C language. The project internally uses c99 features and gnu - extensions, and MSVC is stuck at supporting c89. diff --git a/README.rst b/README.rst new file mode 100644 index 0000000..c481ff5 --- /dev/null +++ b/README.rst @@ -0,0 +1,83 @@ + +Criterion +========= + +.. image:: https://travis-ci.org/Snaipe/Criterion.svg?branch=master + :target: https://travis-ci.org/Snaipe/Criterion +.. image:: https://coveralls.io/repos/Snaipe/Criterion/badge.svg?branch=master + :target: https://coveralls.io/r/Snaipe/Criterion?branch=master +.. image:: https://img.shields.io/badge/license-MIT-blue.svg?style=flat + :target: https://github.com/Snaipe/Criterion/blob/master/LICENSE +.. image:: https://img.shields.io/github/tag/Snaipe/Criterion.svg?label=version&style=flat + :target: https://github.com/Snaipe/Criterion/releases + +A dead-simple, yet extensible, C test framework. + +Philosophy +---------- + +Most test frameworks for C require a lot of boilerplate code to +set up tests and test suites -- you need to create a main, +then register new test suites, then register the tests within +these suits, and finally call the right functions. + +This gives the user great control, at the unfortunate cost of simplicity. + +Criterion follows the KISS principle, while keeping the control +the user would have with other frameworks: + +* Tests are automatically registered when declared. +* A default entry point is provided, no need to declare a main + unless you want to do special handling. +* Test are isolated in their own process, crashes and signals can be + reported and tested. +* Progress and statistics can be followed in real time with report hooks. + +Installation +------------ + +.. code-block:: bash + + $ git clone https://github.com/Snaipe/Criterion.git + $ cd Criterion + $ ./autogen.sh && ./configure && make && sudo make install + +Usage +----- + +Given a test file named test.c, compile it with `-lcriterion`: + +.. code-block:: bash + + $ gcc -o test test.c -lcriterion + +Samples +------- + +Sample tests can be found in the `sample directory `_. + +* `A simple test `_ +* `Using multiple suites `_ +* `Tests with signals `_ +* `Using report hooks `_ + +F.A.Q. +------ + +**Q. What's wrong with other test frameworks?** +A. I worked with CUnit and Check, and I must say that they do their job +very well -- the only thing that bugs me is that setting up a test +suite from scratch is a pain, it should really be simpler. Most +(if not all) high-level languages have test frameworks with automatic +test registration, but all the ones for C require you to set up a +main, manually register suites, then tests. Criterion tries to +fix these shortcomings. + +**Q. Where has this been tested?** +A. Currently, on Linux 2.6.32 and Linux 3.15.7, although it should work on +most \*nix systems. More tests will be added on the build matrix. + +**Q. Will this work under Windows/MSVC?** +A. Windows support with MinGW is coming, but MSVC is a bit of a lost cause +for the C language. The project internally uses c99 features and gnu +extensions, and MSVC is stuck at supporting c89. From 2a0277eb1da246d4595ef86479cb3a35bc20bff7 Mon Sep 17 00:00:00 2001 From: Snaipe Date: Mon, 9 Feb 2015 22:14:51 +0100 Subject: [PATCH 02/15] Updated gitignore for docs --- .gitignore | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/.gitignore b/.gitignore index 2980344..59954bb 100644 --- a/.gitignore +++ b/.gitignore @@ -5,12 +5,11 @@ !*.c !*.h +!*.rst !LICENSE !HEADER -!README.md !ChangeLog -!doc/* !Makefile.am !configure.ac From 591809da40a70482aa70fd9690efbfd80fca1f08 Mon Sep 17 00:00:00 2001 From: Snaipe Date: Mon, 9 Feb 2015 23:43:48 +0100 Subject: [PATCH 03/15] Added Getting started doc --- doc/starter.rst | 106 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 doc/starter.rst diff --git a/doc/starter.rst b/doc/starter.rst new file mode 100644 index 0000000..7b17da6 --- /dev/null +++ b/doc/starter.rst @@ -0,0 +1,106 @@ +Getting started +=============== + +Adding tests +------------ + +Adding tests is done using the ``Test`` macro: + +.. code-block:: c + + #include + + Test(suite_name, test_name) { + // test contents + } + +``suite_name`` and ``test_name`` are the identifiers of the test suite and +the test, respectively. These identifiers must follow the language +identifier format. + +Tests are automatically sorted by suite, then by name using the alphabetical +order. + +Asserting things +---------------- + +Assertions come in two kinds: + +* ``assert*`` are assertions that are fatal to the current test if failed; + in other words, if the condition evaluates to ``false``, the test is + marked as a failure and the execution of the function is aborted. +* ``expect*`` are, in the other hand, assertions that are not fatal to the + test. Execution will continue even if the condition evaluates to + ``false``, but the test will be marked as a failure. + +``assert()`` and ``expect()`` are the most simple kinds of assertions +criterion has to offer. They both take a mandatory condition as a first +parameter, and an optional failure message: + +.. code-block:: c + + #include + #include + + Test(sample, test) { + expect(strlen("Test") == 4, "Expected \"Test\" to have a length of 4."); + expect(strlen("Hello") == 4, "This will always fail, why did I add this?"); + assert(strlen("") == 0); + } + +On top of those, more assertions are available for common operations: + +* ``{assert,expect}Equal(Actual, Expected, [Message])`` +* ``{assert,expect}NotEqual(Actual, Unexpected, [Message])`` +* ``{assert,expect}StringsEqual(Actual, Expected, [Message])`` +* ``{assert,expect}StringsNotEqual(Actual, Unexpected, [Message])`` +* ``{assert,expect}ArraysEqual(Actual, Expected, Size, [Message])`` +* ``{assert,expect}ArraysNotEqual(Actual, Unexpected, Size, [Message])`` + +Initialization and finalization +------------------------------- + +Tests that need some setup and teardown can register functions that will +run before and after the test function: + +.. code-block:: c + + #include + #include + + void setup(void) { + puts("Runs before the test"); + } + + void teardown(void) { + puts("Runs after the test"); + } + + Test(suite_name, test_name, .init = setup, .fini = teardown) { + // test contents + } + +Testing signals +--------------- + +If a test receives a signal, it will by default be marked as a failure. +You can, however, expect a test to only pass if a special kind of signal +is received: + +.. code-block:: c + + #include + #include + #include + + // This test will fail + Test(sample, failing) { + int *ptr = NULL; + *ptr = 42; + } + + // This test will pass + Test(sample, passing, .signal = SIGSEGV) { + int *ptr = NULL; + *ptr = 42; + } From a422d423e5432a50a389a46f1aa14c4b5e73efe2 Mon Sep 17 00:00:00 2001 From: Snaipe Date: Mon, 9 Feb 2015 23:53:34 +0100 Subject: [PATCH 04/15] Added documentation index --- doc/index.rst | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/index.rst diff --git a/doc/index.rst b/doc/index.rst new file mode 100644 index 0000000..094c63c --- /dev/null +++ b/doc/index.rst @@ -0,0 +1,8 @@ +Criterion +========= + +Table of Contents: + + .. toctree:: + :maxdepth: 2 + starter From af2498a7f494a377e8a70a0982a4902b2e4b6bf9 Mon Sep 17 00:00:00 2001 From: Snaipe Date: Tue, 10 Feb 2015 00:01:21 +0100 Subject: [PATCH 05/15] Revert "Converted README to ReStructuredText" This reverts commit 7244cbfcca310a320b1f64966959b41d0dc6eb19. --- README.md | 74 ++++++++++++++++++++++++++++++++++++++++++++++++ README.rst | 83 ------------------------------------------------------ 2 files changed, 74 insertions(+), 83 deletions(-) create mode 100644 README.md delete mode 100644 README.rst diff --git a/README.md b/README.md new file mode 100644 index 0000000..3d367c9 --- /dev/null +++ b/README.md @@ -0,0 +1,74 @@ + +Criterion +========= + +[![Build Status](https://travis-ci.org/Snaipe/Criterion.svg?branch=master)](https://travis-ci.org/Snaipe/Criterion) +[![Coverage Status](https://coveralls.io/repos/Snaipe/Criterion/badge.svg?branch=master)](https://coveralls.io/r/Snaipe/Criterion?branch=master) +[![License](https://img.shields.io/badge/license-MIT-blue.svg?style=flat)](https://github.com/Snaipe/Criterion/blob/master/LICENSE) +[![Version](https://img.shields.io/github/tag/Snaipe/Criterion.svg?label=version&style=flat)](https://github.com/Snaipe/Criterion/releases) + +A dead-simple, yet extensible, C test framework. + +## Philosophy + +Most test frameworks for C require a lot of boilerplate code to +set up tests and test suites -- you need to create a main, +then register new test suites, then register the tests within +these suits, and finally call the right functions. + +This gives the user great control, at the unfortunate cost of simplicity. + +Criterion follows the KISS principle, while keeping the control +the user would have with other frameworks: + +* Tests are automatically registered when declared. +* A default entry point is provided, no need to declare a main + unless you want to do special handling. +* Test are isolated in their own process, crashes and signals can be + reported and tested. +* Progress and statistics can be followed in real time with report hooks. + +## Installation + +```bash +$ git clone https://github.com/Snaipe/Criterion.git +$ cd Criterion +$ ./autogen.sh && ./configure && make && sudo make install +``` + +## Usage + +Given a test file named test.c, compile it with `-lcriterion`: + +```bash +$ gcc -o test test.c -lcriterion +``` + +## Samples + +Sample tests can be found in the [sample directory](https://github.com/Snaipe/Criterion/tree/master/samples). + +* [A simple test](https://github.com/Snaipe/Criterion/blob/master/samples/simple.c) +* [Using multiple suites](https://github.com/Snaipe/Criterion/blob/master/samples/suites.c) +* [Tests with signals](https://github.com/Snaipe/Criterion/blob/master/samples/signal.c) +* [Using report hooks](https://github.com/Snaipe/Criterion/blob/master/samples/report.c) + +## F.A.Q. + +**Q. What's wrong with other test frameworks?** +A. I worked with CUnit and Check, and I must say that they do their job + very well -- the only thing that bugs me is that setting up a test + suite from scratch is a pain, it should really be simpler. Most + (if not all) high-level languages have test frameworks with automatic + test registration, but all the ones for C require you to set up a + main, manually register suites, then tests. Criterion tries to + fix these shortcomings. + +**Q. Where has this been tested?** +A. Currently, on Linux 2.6.32 and Linux 3.15.7, although it should work on + most \*nix systems. More tests will be added on the build matrix. + +**Q. Will this work under Windows/MSVC?** +A. Windows support with MinGW is coming, but MSVC is a bit of a lost cause + for the C language. The project internally uses c99 features and gnu + extensions, and MSVC is stuck at supporting c89. diff --git a/README.rst b/README.rst deleted file mode 100644 index c481ff5..0000000 --- a/README.rst +++ /dev/null @@ -1,83 +0,0 @@ - -Criterion -========= - -.. image:: https://travis-ci.org/Snaipe/Criterion.svg?branch=master - :target: https://travis-ci.org/Snaipe/Criterion -.. image:: https://coveralls.io/repos/Snaipe/Criterion/badge.svg?branch=master - :target: https://coveralls.io/r/Snaipe/Criterion?branch=master -.. image:: https://img.shields.io/badge/license-MIT-blue.svg?style=flat - :target: https://github.com/Snaipe/Criterion/blob/master/LICENSE -.. image:: https://img.shields.io/github/tag/Snaipe/Criterion.svg?label=version&style=flat - :target: https://github.com/Snaipe/Criterion/releases - -A dead-simple, yet extensible, C test framework. - -Philosophy ----------- - -Most test frameworks for C require a lot of boilerplate code to -set up tests and test suites -- you need to create a main, -then register new test suites, then register the tests within -these suits, and finally call the right functions. - -This gives the user great control, at the unfortunate cost of simplicity. - -Criterion follows the KISS principle, while keeping the control -the user would have with other frameworks: - -* Tests are automatically registered when declared. -* A default entry point is provided, no need to declare a main - unless you want to do special handling. -* Test are isolated in their own process, crashes and signals can be - reported and tested. -* Progress and statistics can be followed in real time with report hooks. - -Installation ------------- - -.. code-block:: bash - - $ git clone https://github.com/Snaipe/Criterion.git - $ cd Criterion - $ ./autogen.sh && ./configure && make && sudo make install - -Usage ------ - -Given a test file named test.c, compile it with `-lcriterion`: - -.. code-block:: bash - - $ gcc -o test test.c -lcriterion - -Samples -------- - -Sample tests can be found in the `sample directory `_. - -* `A simple test `_ -* `Using multiple suites `_ -* `Tests with signals `_ -* `Using report hooks `_ - -F.A.Q. ------- - -**Q. What's wrong with other test frameworks?** -A. I worked with CUnit and Check, and I must say that they do their job -very well -- the only thing that bugs me is that setting up a test -suite from scratch is a pain, it should really be simpler. Most -(if not all) high-level languages have test frameworks with automatic -test registration, but all the ones for C require you to set up a -main, manually register suites, then tests. Criterion tries to -fix these shortcomings. - -**Q. Where has this been tested?** -A. Currently, on Linux 2.6.32 and Linux 3.15.7, although it should work on -most \*nix systems. More tests will be added on the build matrix. - -**Q. Will this work under Windows/MSVC?** -A. Windows support with MinGW is coming, but MSVC is a bit of a lost cause -for the C language. The project internally uses c99 features and gnu -extensions, and MSVC is stuck at supporting c89. From 86e3f685260c53204d338c7aff67d437b3dac1e3 Mon Sep 17 00:00:00 2001 From: Snaipe Date: Tue, 10 Feb 2015 00:06:09 +0100 Subject: [PATCH 06/15] Fixed toctree not being properly called --- doc/index.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/index.rst b/doc/index.rst index 094c63c..3020278 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -5,4 +5,5 @@ Table of Contents: .. toctree:: :maxdepth: 2 + starter From 09c56585748926d8e535588beeb1e3a8539c1b60 Mon Sep 17 00:00:00 2001 From: Snaipe Date: Tue, 10 Feb 2015 00:19:51 +0100 Subject: [PATCH 07/15] Added sphinx doc configuration --- doc/conf.py | 292 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 292 insertions(+) create mode 100644 doc/conf.py diff --git a/doc/conf.py b/doc/conf.py new file mode 100644 index 0000000..5cb2107 --- /dev/null +++ b/doc/conf.py @@ -0,0 +1,292 @@ +# -*- coding: utf-8 -*- +import sys +import os + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +#sys.path.insert(0, os.path.abspath('.')) +sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "_ext"))) + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +#needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix of source filenames. +source_suffix = '.rst' + +# The encoding of source files. +#source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = u'Criterion' +copyright = u'2015, Franklin "Snaipe" Mathieu' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version = '0.1.0' +# The full version, including alpha/beta/rc tags. +release = version + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +#language = None + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +#today = '' +# Else, today_fmt is used as the format for a strftime call. +#today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = ['_build', 'html', 'doctrees'] + +# The reST default role (used for this markup: `text`) to use for all +# documents. +#default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +#add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +#add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +#show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# A list of ignored prefixes for module index sorting. +#modindex_common_prefix = [] + +# If true, keep warnings as "system message" paragraphs in the built documents. +#keep_warnings = False + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = 'default' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +#html_theme_options = {} + +# Add any paths that contain custom themes here, relative to this directory. +#html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# " v documentation". +#html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +#html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +#html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +#html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +# Add any extra paths that contain custom files (such as robots.txt or +# .htaccess) here, relative to this directory. These files are copied +# directly to the root of the documentation. +#html_extra_path = [] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +#html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +#html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +#html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +#html_additional_pages = {} + +# If false, no module index is generated. +#html_domain_indices = True + +# If false, no index is generated. +#html_use_index = True + +# If true, the index is split into individual pages for each letter. +#html_split_index = False + +# If true, links to the reST sources are added to the pages. +#html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +#html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +#html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +#html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +#html_file_suffix = None + +# Output file base name for HTML help builder. +htmlhelp_basename = 'CriterionDoc' + + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { +# The paper size ('letterpaper' or 'a4paper'). +#'papersize': 'letterpaper', + +# The font size ('10pt', '11pt' or '12pt'). +#'pointsize': '10pt', + +# Additional stuff for the LaTeX preamble. +#'preamble': '', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + ('index', 'criterion.tex', u'Criterion Documentation', + u'Franklin "Snaipe" Mathieu', 'manual'), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +#latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +#latex_use_parts = False + +# If true, show page references after internal links. +#latex_show_pagerefs = False + +# If true, show URL addresses after external links. +#latex_show_urls = False + +# Documents to append as an appendix to all manuals. +#latex_appendices = [] + +# If false, no module index is generated. +#latex_domain_indices = True + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + ('index', 'criterion', u'Criterion Documentation', + [u'Franklin "Snaipe" Mathieu'], 3) +] + +# If true, show URL addresses after external links. +#man_show_urls = False + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + ('index', 'Criterion', u'Criterion Documentation', + u'Criterion', 'Criterion', '', + ''), +] + +# Documents to append as an appendix to all manuals. +#texinfo_appendices = [] + +# If false, no module index is generated. +#texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +#texinfo_show_urls = 'footnote' + + +# -- Options for Epub output --------------------------------------------------- + +# Bibliographic Dublin Core info. +epub_title = u'Criterion' +epub_author = u'Franklin "Snaipe" Mathieu' +epub_publisher = u'Franklin "Snaipe" Mathieu' +epub_copyright = copyright + +# The language of the text. It defaults to the language option +# or en if the language is not set. +#epub_language = '' + +# The scheme of the identifier. Typical schemes are ISBN or URL. +#epub_scheme = '' + +# The unique identifier of the text. This can be a ISBN number +# or the project homepage. +#epub_identifier = '' + +# A unique identification for the text. +#epub_uid = '' + +# A tuple containing the cover image and cover page html template filenames. +#epub_cover = () + +# HTML files that should be inserted before the pages created by sphinx. +# The format is a list of tuples containing the path and title. +#epub_pre_files = [] + +# HTML files shat should be inserted after the pages created by sphinx. +# The format is a list of tuples containing the path and title. +#epub_post_files = [] + +# A list of files that should not be packed into the epub file. +#epub_exclude_files = [] + +# The depth of the table of contents in toc.ncx. +#epub_tocdepth = 3 + +# Allow duplicate toc entries. +#epub_tocdup = True + +# Highlight PHP without starting Date: Tue, 10 Feb 2015 00:27:58 +0100 Subject: [PATCH 08/15] Added bottom links in doc index --- doc/index.rst | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/doc/index.rst b/doc/index.rst index 3020278..7a59a3a 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -1,9 +1,14 @@ Criterion ========= -Table of Contents: - .. toctree:: :maxdepth: 2 starter + +Indices and tables +================== + +* :ref:`genindex` +* :ref:`search` +* :ref:`glossary` From 7eb03b68d01216d290f397dbd85404d5e85a3203 Mon Sep 17 00:00:00 2001 From: Snaipe Date: Tue, 10 Feb 2015 01:37:45 +0100 Subject: [PATCH 09/15] Added a bunch of docs, removed additional links in index. --- doc/hooks.rst | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ doc/index.rst | 10 +++------- doc/intro.rst | 28 ++++++++++++++++++++++++++++ doc/setup.rst | 29 +++++++++++++++++++++++++++++ 4 files changed, 108 insertions(+), 7 deletions(-) create mode 100644 doc/hooks.rst create mode 100644 doc/intro.rst create mode 100644 doc/setup.rst diff --git a/doc/hooks.rst b/doc/hooks.rst new file mode 100644 index 0000000..1450305 --- /dev/null +++ b/doc/hooks.rst @@ -0,0 +1,48 @@ +Report Hooks +============ + +Report hooks are functions that are called at key moments during the testing +process. These are useful to report statistics gathered during the execution. + +A report hook can be declared using the ``ReportHook`` macro: + +.. code-block:: c + + #include + #include + + ReportHook(Phase)() { + } + +The macro takes a Phase parameter that indicates the phase at which the function +shall be run. Valid phases are described below. + +Testing Phases +-------------- + +The flow of the test process goes as follows: + +1. ``PRE_EVERYTHING``: occurs before running the tests. +#. ``PRE_INIT``: occurs before a test is initialized. +#. ``PRE_TEST``: occurs after the test initialization, but before the test is run. +#. ``ASSERT``: occurs when an assertion is hit +#. ``TEST_CRASH``: occurs when a test crashes unexpectedly. +#. ``POST_TEST``: occurs after a test ends, but before the test finalization. +#. ``POST_FINI``: occurs after a test finalization. +#. ``POST_EVERYTHING``: occurs after all the tests are done. + +Hook Parameters +--------------- + +A report hook may take zero or one parameter. If a parameter is given, it +is undefined behaviour if it is not a pointer type and not of the proper pointed +type for that phase. + +Valid types for each phases are: + +* ``struct criterion_test *`` for ``PRE_INIT`` and ``PRE_TEST``. +* ``struct criterion_test_stats *`` for ``POST_TEST``, ``POST_FINI``, and ``TEST_CRASH``. +* ``struct criterion_assert_stats *`` for ``ASSERT``. +* ``struct criterion_global_stats *`` for ``POST_EVERYTHING``. + +``PRE_EVERYTHING`` does not take any parameter. diff --git a/doc/index.rst b/doc/index.rst index 7a59a3a..7aba9ad 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -4,11 +4,7 @@ Criterion .. toctree:: :maxdepth: 2 + intro + setup starter - -Indices and tables -================== - -* :ref:`genindex` -* :ref:`search` -* :ref:`glossary` + hooks diff --git a/doc/intro.rst b/doc/intro.rst new file mode 100644 index 0000000..e26f031 --- /dev/null +++ b/doc/intro.rst @@ -0,0 +1,28 @@ +Introduction +============ + +Criterion is a dead-simple, non-intrusive testing framework for the C +programming language. + +Philosophy +---------- + +Most test frameworks for C require a lot of boilerplate code to +set up tests and test suites -- you need to create a main, +then register new test suites, then register the tests within +these suits, and finally call the right functions. + +This gives the user great control, at the unfortunate cost of simplicity. + +Criterion follows the KISS principle, while keeping the control +the user would have with other frameworks. + +Features +-------- + +* Tests are automatically registered when declared. +* A default entry point is provided, no need to declare a main + unless you want to do special handling. +* Test are isolated in their own process, crashes and signals can be + reported and tested. +* Progress and statistics can be followed in real time with report hooks. diff --git a/doc/setup.rst b/doc/setup.rst new file mode 100644 index 0000000..aa26708 --- /dev/null +++ b/doc/setup.rst @@ -0,0 +1,29 @@ +Setup +===== + +Prerequisites +------------- + +Currently, this library only works under \*nix systems. + +To compile the static library and its dependencies, GCC 4.9+ is needed. + +To use the static library, GCC or Clang are needed. + +Installation +------------ + +.. code-block:: bash + + $ git clone https://github.com/Snaipe/Criterion.git + $ cd Criterion + $ ./autogen.sh && ./configure && make && sudo make install + +Usage +----- + +Given a test file named test.c, compile it with `-lcriterion`: + +.. code-block:: bash + + $ gcc -o test test.c -lcriterion From d4eb936c00f20730eedab986075ce53e7a8f6030 Mon Sep 17 00:00:00 2001 From: Snaipe Date: Tue, 10 Feb 2015 15:10:48 +0100 Subject: [PATCH 10/15] Added valgrind-friendly no-early-exit for leak checks --- src/runner.c | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/src/runner.c b/src/runner.c index 40b63b9..5d7afc3 100644 --- a/src/runner.c +++ b/src/runner.c @@ -35,6 +35,8 @@ static struct criterion_test * const g_section_start = &__start_criterion_tests; static struct criterion_test * const g_section_end = &__stop_criterion_tests; +static pid_t g_runner_pid; + struct test_set { struct criterion_test **tests; size_t nb_tests; @@ -87,8 +89,11 @@ static struct test_set *read_all_tests(void) { static void map_tests(struct test_set *set, struct criterion_global_stats *stats, void (*fun)(struct criterion_global_stats *, struct criterion_test *)) { size_t i = 0; - for (struct criterion_test **t = set->tests; i < set->nb_tests; ++i, ++t) + for (struct criterion_test **t = set->tests; i < set->nb_tests; ++i, ++t) { fun(stats, *t); + if (g_runner_pid != getpid()) + return; + } } __attribute__ ((always_inline)) @@ -126,7 +131,10 @@ static void run_test(struct criterion_global_stats *stats, struct criterion_test setup_child(&fds); run_test_child(test); - _exit(0); + if (!strcmp("1", getenv("CRITERION_NO_EARLY_EXIT") ?: "0")) + return; + else + _exit(0); } else { close(fds.out); struct event *ev; @@ -163,16 +171,28 @@ static void run_test(struct criterion_global_stats *stats, struct criterion_test } // TODO: disable & change tests at runtime -int criterion_run_all_tests(void) { +static int criterion_run_all_tests_impl(void) { report(PRE_EVERYTHING, NULL); + g_runner_pid = getpid(); smart struct test_set *set = read_all_tests(); smart struct criterion_global_stats *stats = stats_init(); if (!set) abort(); map_tests(set, stats, run_test); - report(POST_EVERYTHING, stats); - return strcmp("1", getenv("CRITERION_ALWAYS_SUCCEED") ?: "0") && stats->tests_failed > 0; + if (g_runner_pid != getpid()) + return -1; + + report(POST_EVERYTHING, stats); + return stats->tests_failed > 0; +} + +int criterion_run_all_tests(void) { + int res = criterion_run_all_tests_impl(); + if (res == -1) // if this is the test worker terminating + exit(0); + + return strcmp("1", getenv("CRITERION_ALWAYS_SUCCEED") ?: "0") && res; } int main(void) { From df1f23ef4cf121e1bdeee2a66428c28885e3835d Mon Sep 17 00:00:00 2001 From: Snaipe Date: Tue, 10 Feb 2015 18:20:06 +0100 Subject: [PATCH 11/15] Refactored process management into its own file --- Makefile.am | 2 + src/process.c | 71 +++++++++++++++++++++++++++++++++++ src/process.h | 23 ++++++++++++ src/runner.c | 102 ++++++++++++++++++-------------------------------- 4 files changed, 132 insertions(+), 66 deletions(-) create mode 100644 src/process.c create mode 100644 src/process.h diff --git a/Makefile.am b/Makefile.am index 684c818..fc4f8b2 100644 --- a/Makefile.am +++ b/Makefile.am @@ -37,5 +37,7 @@ libcriterion_la_SOURCES = \ src/report.h \ src/runner.c \ src/runner.h \ + src/process.c \ + src/process.h \ src/stats.c \ src/stats.h diff --git a/src/process.c b/src/process.c new file mode 100644 index 0000000..3cbab45 --- /dev/null +++ b/src/process.c @@ -0,0 +1,71 @@ +#include +#include +#include +#include +#include +#include + +#include "criterion/criterion.h" +#include "process.h" +#include "event.h" + +struct process { + pid_t pid; + int in; +}; + +static pid_t g_runner_pid; + +void set_runner_pid(void) { + g_runner_pid = getpid(); +} + +bool is_runner(void) { + return g_runner_pid == getpid(); +} + +static void close_process(void *ptr, UNUSED void *meta) { + close(((struct process *) ptr)->in); +} + +struct event *worker_read_event(struct process *proc) { + return read_event(proc->in); +} + +struct process *spawn_test_worker(struct criterion_test *test, void (*func)(struct criterion_test *)) { + int fds[2]; + if (pipe(fds) == -1) + abort(); + + pid_t pid = fork(); + if (pid == -1) { + return NULL; + } else if (!pid) { + close(STDIN_FILENO); + close(fds[0]); + EVENT_PIPE = fds[1]; + + func(test); + close(fds[1]); + if (!strcmp("1", getenv("CRITERION_NO_EARLY_EXIT") ?: "0")) + return NULL; + else + _exit(0); + } + + close(fds[1]); + return unique_ptr(struct process, ({ .pid = pid, .in = fds[0] }), close_process); +} + +struct process_status wait_proc(struct process *proc) { + int status; + waitpid(proc->pid, &status, 0); + + if (WIFEXITED(status)) + return (struct process_status) { .kind = EXIT_STATUS, .status = WEXITSTATUS(status) }; + + if (WIFSIGNALED(status)) + return (struct process_status) { .kind = SIGNAL, .status = WTERMSIG(status) }; + + return (struct process_status) { .kind = STOPPED }; +} diff --git a/src/process.h b/src/process.h new file mode 100644 index 0000000..9eb3358 --- /dev/null +++ b/src/process.h @@ -0,0 +1,23 @@ +#ifndef PROCESS_H_ +# define PROCESS_H_ + +struct process; + +enum status_kind { + EXIT_STATUS, + STOPPED, + SIGNAL, +}; + +struct process_status { + enum status_kind kind; + int status; +}; + +void set_runner_pid(void); +bool is_runner(void); +struct process_status wait_proc(struct process *proc); +struct process *spawn_test_worker(struct criterion_test *test, void (*func)(struct criterion_test *)); +struct event *worker_read_event(struct process *proc); + +#endif /* !PROCESS_H_ */ diff --git a/src/runner.c b/src/runner.c index 5d7afc3..ddc30fe 100644 --- a/src/runner.c +++ b/src/runner.c @@ -31,35 +31,26 @@ #include "runner.h" #include "report.h" #include "event.h" +#include "process.h" static struct criterion_test * const g_section_start = &__start_criterion_tests; static struct criterion_test * const g_section_end = &__stop_criterion_tests; -static pid_t g_runner_pid; - struct test_set { struct criterion_test **tests; size_t nb_tests; }; -static int compare_test_by_name(const struct criterion_test *first, - const struct criterion_test *second) { - // likely to happen - if (first->name == second->name) - return 0; - return strcmp(first->name, second->name); -} - static int compare_test(const void *a, const void *b) { struct criterion_test *first = *(struct criterion_test **) a; struct criterion_test *second = *(struct criterion_test **) b; // likely to happen if (first->category == second->category) { - return compare_test_by_name(first, second); + return strcmp(first->name, second->name); } else { return strcmp(first->category, second->category) - ?: compare_test_by_name(first, second); + ?: strcmp(first->name, second->name); } } @@ -91,7 +82,7 @@ static void map_tests(struct test_set *set, struct criterion_global_stats *stats size_t i = 0; for (struct criterion_test **t = set->tests; i < set->nb_tests; ++i, ++t) { fun(stats, *t); - if (g_runner_pid != getpid()) + if (!is_runner()) return; } } @@ -109,63 +100,41 @@ static void run_test_child(struct criterion_test *test) { send_event(POST_FINI, NULL, 0); } -struct pipefds { - int in, out; -} __attribute__ ((packed)); - -static void setup_child(struct pipefds *fds) { - close(STDIN_FILENO); - close(fds->in); - EVENT_PIPE = fds->out; -} - static void run_test(struct criterion_global_stats *stats, struct criterion_test *test) { smart struct criterion_test_stats *test_stats = test_stats_init(test); - struct pipefds fds; - if (pipe((int*) &fds) == -1) - abort(); + smart struct process *proc = spawn_test_worker(test, run_test_child); + if (proc == NULL && !is_runner()) + return; - pid_t pid; - if (!(pid = fork())) { - setup_child(&fds); - - run_test_child(test); - if (!strcmp("1", getenv("CRITERION_NO_EARLY_EXIT") ?: "0")) - return; - else - _exit(0); - } else { - close(fds.out); - struct event *ev; - while ((ev = read_event(fds.in)) != NULL) { - stat_push_event(stats, test_stats, ev); - switch (ev->kind) { - case PRE_INIT: report(PRE_INIT, test); break; - case PRE_TEST: report(PRE_TEST, test); break; - case ASSERT: report(ASSERT, ev->data); break; - case POST_TEST: report(POST_TEST, test_stats); break; - case POST_FINI: report(POST_FINI, test_stats); break; - } - sfree(ev); + struct event *ev; + while ((ev = worker_read_event(proc)) != NULL) { + stat_push_event(stats, test_stats, ev); + switch (ev->kind) { + case PRE_INIT: report(PRE_INIT, test); break; + case PRE_TEST: report(PRE_TEST, test); break; + case ASSERT: report(ASSERT, ev->data); break; + case POST_TEST: report(POST_TEST, test_stats); break; + case POST_FINI: report(POST_FINI, test_stats); break; } - int status; - waitpid(pid, &status, 0); - if (WIFSIGNALED(status)) { - test_stats->signal = WTERMSIG(status); - if (test->data->signal == 0) { - struct event ev = { .kind = TEST_CRASH }; - stat_push_event(stats, test_stats, &ev); - report(TEST_CRASH, test_stats); - } else { - struct event ev = { .kind = POST_TEST }; - stat_push_event(stats, test_stats, &ev); - report(POST_TEST, test_stats); + sfree(ev); + } - ev.kind = POST_FINI; - stat_push_event(stats, test_stats, &ev); - report(POST_FINI, test_stats); - } + struct process_status status = wait_proc(proc); + if (status.kind == SIGNAL) { + test_stats->signal = status.status; + if (test->data->signal == 0) { + struct event ev = { .kind = TEST_CRASH }; + stat_push_event(stats, test_stats, &ev); + report(TEST_CRASH, test_stats); + } else { + struct event ev = { .kind = POST_TEST }; + stat_push_event(stats, test_stats, &ev); + report(POST_TEST, test_stats); + + ev.kind = POST_FINI; + stat_push_event(stats, test_stats, &ev); + report(POST_FINI, test_stats); } } } @@ -173,14 +142,15 @@ static void run_test(struct criterion_global_stats *stats, struct criterion_test // TODO: disable & change tests at runtime static int criterion_run_all_tests_impl(void) { report(PRE_EVERYTHING, NULL); - g_runner_pid = getpid(); + set_runner_pid(); + smart struct test_set *set = read_all_tests(); smart struct criterion_global_stats *stats = stats_init(); if (!set) abort(); map_tests(set, stats, run_test); - if (g_runner_pid != getpid()) + if (!is_runner()) return -1; report(POST_EVERYTHING, stats); From 7e2d485d7522d2190f29028cb8618ebeee69fb4a Mon Sep 17 00:00:00 2001 From: Snaipe Date: Wed, 11 Feb 2015 01:25:48 +0100 Subject: [PATCH 12/15] Moved main function in its own CU to avoid symbol conflicts when calling other functions --- Makefile.am | 3 ++- src/main.c | 5 +++++ src/runner.c | 4 ---- 3 files changed, 7 insertions(+), 5 deletions(-) create mode 100644 src/main.c diff --git a/Makefile.am b/Makefile.am index fc4f8b2..00a090d 100644 --- a/Makefile.am +++ b/Makefile.am @@ -40,4 +40,5 @@ libcriterion_la_SOURCES = \ src/process.c \ src/process.h \ src/stats.c \ - src/stats.h + src/stats.h \ + src/main.c diff --git a/src/main.c b/src/main.c new file mode 100644 index 0000000..bcaac01 --- /dev/null +++ b/src/main.c @@ -0,0 +1,5 @@ +#include + +int main(void) { + return criterion_run_all_tests(); +} diff --git a/src/runner.c b/src/runner.c index ddc30fe..e00d4fe 100644 --- a/src/runner.c +++ b/src/runner.c @@ -164,7 +164,3 @@ int criterion_run_all_tests(void) { return strcmp("1", getenv("CRITERION_ALWAYS_SUCCEED") ?: "0") && res; } - -int main(void) { - return criterion_run_all_tests(); -} From ece20af1bc4b4f56bce0e1f2d7d9d798a00bbd1e Mon Sep 17 00:00:00 2001 From: Snaipe Date: Wed, 11 Feb 2015 13:34:46 +0100 Subject: [PATCH 13/15] Updated readme with docs, removed installation process, added footer links --- README.md | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index 3d367c9..7130d9f 100644 --- a/README.md +++ b/README.md @@ -28,30 +28,19 @@ the user would have with other frameworks: reported and tested. * Progress and statistics can be followed in real time with report hooks. -## Installation +## Documentation -```bash -$ git clone https://github.com/Snaipe/Criterion.git -$ cd Criterion -$ ./autogen.sh && ./configure && make && sudo make install -``` - -## Usage - -Given a test file named test.c, compile it with `-lcriterion`: - -```bash -$ gcc -o test test.c -lcriterion -``` +An online documentation is available on [ReadTheDocs][online-docs] +([PDF][pdf-docs] | [Zip][zip-docs] | [Epub][epub-docs]) ## Samples -Sample tests can be found in the [sample directory](https://github.com/Snaipe/Criterion/tree/master/samples). +Sample tests can be found in the [sample directory][samples]. -* [A simple test](https://github.com/Snaipe/Criterion/blob/master/samples/simple.c) -* [Using multiple suites](https://github.com/Snaipe/Criterion/blob/master/samples/suites.c) -* [Tests with signals](https://github.com/Snaipe/Criterion/blob/master/samples/signal.c) -* [Using report hooks](https://github.com/Snaipe/Criterion/blob/master/samples/report.c) +* [A simple test][sample-simple] +* [Using multiple suites][sample-suites] +* [Tests with signals][sample-signal] +* [Using report hooks][sample-report] ## F.A.Q. @@ -72,3 +61,14 @@ A. Currently, on Linux 2.6.32 and Linux 3.15.7, although it should work on A. Windows support with MinGW is coming, but MSVC is a bit of a lost cause for the C language. The project internally uses c99 features and gnu extensions, and MSVC is stuck at supporting c89. + +[online-docs]: http://criterion.readthedocs.org/ +[pdf-docs]: http://readthedocs.org/projects/criterion/downloads/pdf/latest/ +[zip-docs]: http://readthedocs.org/projects/criterion/downloads/htmlzip/latest/ +[epub-docs]: http://readthedocs.org/projects/criterion/downloads/epub/latest/ + +[samples]: https://github.com/Snaipe/Criterion/tree/master/samples +[sample-simple]: https://github.com/Snaipe/Criterion/blob/master/samples/simple.c +[sample-suites]: https://github.com/Snaipe/Criterion/blob/master/samples/suites.c +[sample-signal]: https://github.com/Snaipe/Criterion/blob/master/samples/signal.c +[sample-report]: https://github.com/Snaipe/Criterion/blob/master/samples/report.c From ae9b42a911f30d1666e5f3cc82ce28eed7afaebf Mon Sep 17 00:00:00 2001 From: Snaipe Date: Wed, 11 Feb 2015 14:22:49 +0100 Subject: [PATCH 14/15] Added environment variable documentation --- doc/env.rst | 15 +++++++++++++++ doc/index.rst | 1 + 2 files changed, 16 insertions(+) create mode 100644 doc/env.rst diff --git a/doc/env.rst b/doc/env.rst new file mode 100644 index 0000000..091f489 --- /dev/null +++ b/doc/env.rst @@ -0,0 +1,15 @@ +Environment and CLI +=================== + +Tests built with Criterion support environment variables to alter +their runtime behaviour. + +Environment Variables +--------------------- + +* `CRITERION_ALWAYS_SUCCEED`: when set to `1`, the exit status of the test + process will be 0, regardless if the tests failed or not. +* `CRITERION_NO_EARLY_EXIT`: when set to `1`, the test workers shall not + call `_exit` when done and will properly return from the main and + clean up their process space. This is useful when tracking memory leaks with + `valgrind --tool=memcheck`. diff --git a/doc/index.rst b/doc/index.rst index 7aba9ad..b138762 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -8,3 +8,4 @@ Criterion setup starter hooks + env From 864c892eac23d5c3481af1443f4366d13d95a384 Mon Sep 17 00:00:00 2001 From: Snaipe Date: Wed, 11 Feb 2015 14:25:39 +0100 Subject: [PATCH 15/15] Rewording FAQ answer on windows --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 7130d9f..d04edfb 100644 --- a/README.md +++ b/README.md @@ -58,9 +58,9 @@ A. Currently, on Linux 2.6.32 and Linux 3.15.7, although it should work on most \*nix systems. More tests will be added on the build matrix. **Q. Will this work under Windows/MSVC?** -A. Windows support with MinGW is coming, but MSVC is a bit of a lost cause - for the C language. The project internally uses c99 features and gnu - extensions, and MSVC is stuck at supporting c89. +A. Windows support with MinGW/MSVC is coming, but MSVC is a bit of a lost cause + to compile the library itself: the project internally uses c99 features and gnu + extensions. [online-docs]: http://criterion.readthedocs.org/ [pdf-docs]: http://readthedocs.org/projects/criterion/downloads/pdf/latest/